Advanced Web Scraping: Bypassing "403 Forbidden," captchas, and more
"It just seems like many of the things that I work on require me to get my hands on data that isn’t available any other way"
http://sangaline.com/post/advanced-web- ... -tutorial/
pretty sweet write-up, and a good tool
https://doc.scrapy.org/en/latest/intro/tutorial.html
Advanced Web Scraping
Advanced Web Scraping
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:23.0) Gecko/20130410 Firefox/23.0