As more sites are leaning on JavaScript to load dynamic data, web scraping is hitting new hurdles. Purely using the urllib2 library from Python will return the site without the dynamic data available to gather if a JavaScript library is used to populate site information. Hitting this wall, I decided to take a deep diveContinue reading “Using Python to scrape website data loaded through JavaScript”
Category Archives: Software Development
Building a digital newspaper machine
Awhile back, a friend of mine, Nate Papes, ran across Scott Walker, director of enterprise and investigation at Alabama Media Group, who had retrofitted a newspaper machine to display newspaper front pages digitally. (Update March 13, 2016: This link is dead; it appears Walker removed the post.) We talked about creating our own, but neverContinue reading “Building a digital newspaper machine”