Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Scraping Data From Interactive Map

I would like to scrape the voter registration data underlying this map:

http://www.bostonglobe.com/metro/2012/08/28/registration-figures-show-massachusetts-voters-continue-abandon-two-major-political-parties/p0zW7Snj9R07DK913P36kM/igraphic.html?p1=Article_Graphic

As you hover over each town, both the total and the by-party figures in the box below change. I would like to record the name of each town and registration counts by party. Any suggestions about how to do this would be very helpful. I've done some scraping in the past but not with anything interactive like this and my first few attempts with various python packages haven't worked.

(Unfortunately, the sources link at the bottom is not useful because it just links to the Secretary of State's main page and the original report with data is no longer available)

like image 232
jfeigenbaum Avatar asked May 29 '15 15:05

jfeigenbaum


1 Answers

No need to build an app to scrape this, I just looked at the source of the page for 5 min, check this out:

http://www.bostonglobe.com/r/Boston/2011-2020/WebGraphics/Metro/BostonGlobe.com/2012/08/voter/voter-regs.js

Otherwise

You need to have a technology that has a javascript engine, like phantomjs or firefox/chrome drivers.

If you want to stick to python, Selenium webdriver is the way to go.

http://www.seleniumhq.org/projects/webdriver/

like image 194
Maresh Avatar answered Oct 23 '22 04:10

Maresh