I am attempting to scrape the below website:
http://www.fangraphs.com/leaders.aspx?pos=all&stats=bat&lg=all&qual=0&type=8&season=2011&month=0&season1=2011&ind=0&team=0&rost=0&players=0
If you click the small button at the top-right of the table titled "export data", a javascript script runs and my browser downloads the file in .csv form. I'd like to be able to write a PhantomJS script that can do this automatically. Any ideas?
The above button is coded into HTML as such:
<a id="LB_cmdCSV" href="javascript:__doPostBack('LB$cmdCSV','')">Export Data</a></div>
I also found this function in the HTML source code:
<script type="text/javascript">
//<![CDATA[
var theForm = document.forms['form1'];
if (!theForm) {
theForm = document.form1;
}
function __doPostBack(eventTarget, eventArgument) {
if (!theForm.onsubmit || (theForm.onsubmit() != false)) {
theForm.__EVENTTARGET.value = eventTarget;
theForm.__EVENTARGUMENT.value = eventArgument;
theForm.submit();
}
}
//]]>
</script>
I'm very new to PhantomJS/Javascript and could use some pointers here. I think I've found all the info I need to do this automatically (correct me if I'm wrong), but just not sure where to start on coding it. Thanks for any help.
EDIT - This is what my script looks like right now:
var page = new WebPage();
url = 'http://www.fangraphs.com/leaders.aspx?pos=all&stats=bat&lg=all&qual=0&type=8&season=2011&month=0&season1=2011&ind=0&team=0&rost=0& players=0';
page.open(encodeURI(url), function (status){
if (status !== "success") {
console.log("Unable to access website");
} else {
page.evaluate(function() {
__doPostBack('LB$cmdCSV', '');
});
}
phantom.exit(0);
});
open the Js script link and press ctrl+s i.e save it. Save it by any desired name and then copy it your project folder and then include it in your project files where you have included other files like jquery and css.
Using a Custom-Written Function to Create and Download Text Files in JavaScript. This approach will create text data on the fly and then use JavaScript to create a text file and then download it.
What have worked very well for me is simulating mouse clicks on the desired element.
page.evaluate(function () {
var btn = document.getElementById('LB_cmdCSV')
var ev = document.createEvent('MouseEvent')
ev.initEvent('click', true, true)
btn.dispatchEvent(ev)
})
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With