Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

PhantomJS download using a javascript link

I am attempting to scrape the below website:

http://www.fangraphs.com/leaders.aspx?pos=all&stats=bat&lg=all&qual=0&type=8&season=2011&month=0&season1=2011&ind=0&team=0&rost=0&players=0

If you click the small button at the top-right of the table titled "export data", a javascript script runs and my browser downloads the file in .csv form. I'd like to be able to write a PhantomJS script that can do this automatically. Any ideas?

The above button is coded into HTML as such:

<a id="LB_cmdCSV" href="javascript:__doPostBack('LB$cmdCSV','')">Export Data</a></div>

I also found this function in the HTML source code:

<script type="text/javascript">
//<![CDATA[
var theForm = document.forms['form1'];
if (!theForm) {
    theForm = document.form1;
}
function __doPostBack(eventTarget, eventArgument) {
    if (!theForm.onsubmit || (theForm.onsubmit() != false)) {
        theForm.__EVENTTARGET.value = eventTarget;
        theForm.__EVENTARGUMENT.value = eventArgument;
        theForm.submit();
    }
}
//]]>
</script>

I'm very new to PhantomJS/Javascript and could use some pointers here. I think I've found all the info I need to do this automatically (correct me if I'm wrong), but just not sure where to start on coding it. Thanks for any help.

EDIT - This is what my script looks like right now:

var page = new WebPage();
url = 'http://www.fangraphs.com/leaders.aspx?pos=all&stats=bat&lg=all&qual=0&type=8&season=2011&month=0&season1=2011&ind=0&team=0&rost=0& players=0';

page.open(encodeURI(url), function (status){
  if (status !== "success") {
    console.log("Unable to access website");
  } else {
      page.evaluate(function() {
        __doPostBack('LB$cmdCSV', '');
      });
    }
  phantom.exit(0);
});
like image 419
Duke Silver Avatar asked Jan 19 '12 19:01

Duke Silver


People also ask

How do I download a JavaScript file as a link?

open the Js script link and press ctrl+s i.e save it. Save it by any desired name and then copy it your project folder and then include it in your project files where you have included other files like jquery and css.

Can JavaScript download a file?

Using a Custom-Written Function to Create and Download Text Files in JavaScript. This approach will create text data on the fly and then use JavaScript to create a text file and then download it.


1 Answers

What have worked very well for me is simulating mouse clicks on the desired element.

page.evaluate(function () {
  var btn = document.getElementById('LB_cmdCSV')
  var ev = document.createEvent('MouseEvent')
  ev.initEvent('click', true, true)
  btn.dispatchEvent(ev)
})
like image 155
Linus Unnebäck Avatar answered Oct 12 '22 23:10

Linus Unnebäck