Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How do I set proxy in phantomjs

This https://www.npmjs.com/package/phantom#functionality-details page says:

You can also pass command line switches to the phantomjs process by specifying additional args to phantom.create(), eg:

phantom.create '--load-images=no', '--local-to-remote-url-access=yes', (page) ->

or by specifying them in the options* object:

phantom.create {parameters: {'load-images': 'no', 'local-to-remote-url-access': 'yes'}}, (page) ->

These examples are only in coffee script and also they insinuate that the create function can take

create('string',function)

or

create([object object],function)

but really the first parameter expected is the function!

I really wanted to try http://phantomjs.org/api/command-line.html I might have the wrong idea but to me it looks like they can be used in the create function (right before you do the createPage), am I wrong?

I have tried several things, the most logical one is this:

var phantom = require('phantom');
phantom.create(function(browser){
    browser.createPage(function(page){
        page.open('http://example.com/req.php', function() {

            });},{parameters:{'proxy':'98.239.198.83:21320'}});});

So the page gets opened. I know this because I am making req.php save the $_SERVER object to a txt pad but, the REMOTE_ADDR and REMOTE_PORT headers are not the ones in the proxy I have set. They have no effect. I have also tried:

{options:{'proxy':'98.239.198.83:21320'}}

As the docs call that object the options* object *see above^

and

'--proxy=98.239.198.83:21320'

I have also had a dig through the phantom module to find the create function. It is not written in js I can't see it at least. It must be in C++. It looks like this module has been updated but, the examples deep inside the module look like old code.

How do I do this?

EDIT:

var phantom = require('phantom');
phantom.create(function(browser){
    browser.createPage(function(page){

    browser.setProxy('98.239.198.83','21320','http', null, null, function(){

    page.open(
        'http://example.com/req.php', function() {

         });});});});

This produces no error and the page gets scraped but the proxy is ignored.

like image 747
Ben Muircroft Avatar asked Feb 17 '15 21:02

Ben Muircroft


2 Answers

As for as phantom 2.0.10 version the following code is running very well in my windows machine

  phantom.create(["--proxy=201.172.242.184:15124", "--proxy-type=socks5"])
      .then((instance) => {
          phInstance = instance;
          return instance.createPage();
      })
      .then((page) => {
          sitepage = page;
          return page.open('http://newsdaily.online');
      })
      .then((status) => {
          console.log(status);
          return sitepage.property('title');
      })
      .then((content) => {
          console.log(content);
          sitepage.close();
          phInstance.exit();
      })
      .catch((error) => {
          console.log(error);
          phInstance.exit();
      });
like image 113
Suben Saha Avatar answered Oct 04 '22 16:10

Suben Saha


{ parameters: { 'proxy': 'socks://98.239.198.83:21320' } }

They didn't update their docs.

like image 40
Ben Muircroft Avatar answered Oct 04 '22 15:10

Ben Muircroft