120 bookmarks. First posted by awiedmer march 2017.
I wouldn’t really consider web scraping one of my hobbies or anything but I guess I sort of do a lot of it. It just seems like many of the things that I work on require me to get my hands on data that isn’t available any other way. I need to do static analysis of games for Intoli and so I scrape the Google Play Store to find new ones and download the apks. The Pointy Ball extension requires aggregating fantasy football projections from various sites and the easiest way was to write a scraper. When I think about it, I’ve probably written about 40-50 scrapers. I’m not quite at the point where I’m lying to my family about how many terabytes of data I’m hoarding away… but I’m close.Python Web_Scraping
march 2017 by GameGamer43
tags20 26 @issues @python @tips @web_scraping archive articles blog_posts captcha captchas crawl crawler datascience dev favs framework:scrapy howto html interesting knowledgeliberation lang:python learn min minutes muckrock_feature_ideas online perl programming-general programming-python programming python python_web_crawler pytr-2017-03 read-later s science scrape scraper scraping scrapy search security spider tips tooling tutorial type:tutorial via:popular web-scrape web-scraping web-scrapping web webscraper webscraping web_scraping wrk-tec