Error running PhantomJS with Selenium RemoteWebDriver

Question

I start a selenium grid hub with it's default startup command,

java -jar selenium-server-standalone-2.33.0.jar -role hub

And I start up PhantomJS in it's webdriver mode, on the same machine, like,

phantomjs --webdriver=8080 --webdriver-selenium-grid-hub=http://127.0.0.1:4444

When PhantomJS starts up, I get nothing but "All 'OK'" messages,

[INFO  - 2013-09-09T18:10:38.849Z] GhostDriver - Main - running on port 8080
[INFO  - 2013-09-09T18:10:38.850Z] GhostDriver - Main - registering to Selenium HUB 'http://127.0.0.1:4444' using '127.0.0.1:8080'
[INFO  - 2013-09-09T18:11:47.164Z] HUB Register - register - Registered with grid hub: http://127.0.0.1:4444/ (ok)

But if I try to use that browser, with

driver = webdriver.Remote(command_executor='http://127.0.0.1:4444/wd/hub',
                          desired_capabilities=DesiredCapabilities.PHANTOMJS)

Looking at the output from the PhantomJS executable,

[INFO  - 2013-09-09T18:17:12.531Z] Session [0c5c9500-197c-11e3-8eed-b3b7a73965aa] - _decorateNewWindow - page.settings: {"XSSAuditingEnabled":false,"javascriptCanCloseWindows":true,"javascriptCanOpenWindows":true,"javascriptEnabled":true,"loadImages":true,"localToRemoteUrlAccessEnabled":false,"userAgent":"Mozilla/5.0 (Macintosh; Intel Mac OS X) AppleWebKit/534.34 (KHTML, like Gecko) PhantomJS/1.9.2 Safari/534.34","webSecurityEnabled":true}
[INFO  - 2013-09-09T18:17:12.531Z] Session [0c5c9500-197c-11e3-8eed-b3b7a73965aa] - page.customHeaders:  - {}
[INFO  - 2013-09-09T18:17:12.531Z] Session [0c5c9500-197c-11e3-8eed-b3b7a73965aa] - CONSTRUCTOR - Desired Capabilities: {"platform":"ANY","javascriptEnabled":true,"browserName":"phantomjs","version":""}
[INFO  - 2013-09-09T18:17:12.531Z] Session [0c5c9500-197c-11e3-8eed-b3b7a73965aa] - CONSTRUCTOR - Negotiated Capabilities: {"browserName":"phantomjs","version":"1.9.2","driverName":"ghostdriver","driverVersion":"1.0.4","platform":"mac-10.7 (Lion)-32bit","javascriptEnabled":true,"takesScreenshot":true,"handlesAlerts":false,"databaseEnabled":false,"locationContextEnabled":false,"applicationCacheEnabled":false,"browserConnectionEnabled":false,"cssSelectorsEnabled":true,"webStorageEnabled":false,"rotatable":false,"acceptSslCerts":false,"nativeEvents":true,"proxy":{"proxyType":"direct"}}
[INFO  - 2013-09-09T18:17:12.531Z] SessionManagerReqHand - _postNewSessionCommand - New Session Created: 0c5c9500-197c-11e3-8eed-b3b7a73965aa

Everything seems to be fine, but the code crashes :/ I get a huge error stacktrace, mainly down to,

selenium.common.exceptions.WebDriverException: Message: u'Error forwarding the new session new session request for webdriver should contain a location header with the session.'

And if you check the output of the grid hub,

INFO: Trying to create a new session on test slot {seleniumProtocol=WebDriver, browserName=phantomjs, maxInstances=1}
INFO: Available nodes: [host :http://127.0.0.1:8080 time out : 300000]

There also seems to be an issue. It seems like, for whatever reason, the grid hub doesn't get any information back from the PhantomJS executable, even though it seems like it is processing things correctly.

For testing, I also have a regular selenium node running, on the same machine, with the command

java -jar selenium-server-standalone-2.33.0.jar -role node

and if I start up a Firefox RemoteWebDriver session (in Python), with

driver = webdriver.Remote(command_executor='http://127.0.0.1:4444/wd/hub',
                          desired_capabilities=DesiredCapabilities.FIREFOX)

all is fine.

It looks like everything is set up correctly, I'm keeping things as 'vanilla' and close to what the docs say as possible, but just can't get over the fence here.

The exact same error happens when trying in Java, too

WebDriver driver = new RemoteWebDriver(DesiredCapabilities.phantomjs());

Screenshot of the grid console with both connected, PhantomJS is alive and well there,

Left is default selenium node, right is PhantomJS (and 10.0.0.222 is my laptop's IP address in work).

And my versions are 1.9.2 for PhantomJS and obviously 2.33.0 for selenium.

Answer 1

Turns out that this is a bug feature in the 1.9.2 version of phantomjs. They updated to the latest ghostdriver (v1.4) library and I imagine that is where it was introduced. Ghostdriver is adhering to the new Selenium protocol using posts for creating sessions and not utilizing a location header. Selenium now grabs the location etc from the json in the body of the post. As it turns out the error you are seeing and that I had been seeing is because you are using the old selenium jar which uses the old style session creation protocol with the new version of selenium. For now the answer to your question is to go back to phantomjs 1.9.1 to update your selenium jar to 2.35.0 (see below for link). To reproduce your error I tested with both the ruby and python selenium webdrivers and the 2.33.0 selenium jar running as hub and phantomjs 1.9.2 running as a ghostdriver worker.

Here is the same error described in a github issue:

https://github.com/ariya/phantomjs/issues/11610

I believe it has to do with this particular commit. You can see the header for "Location" has been removed.

https://github.com/detro/ghostdriver/commit/60bf3c90aa825cc3db4fa899b6a67bb7f109d55d

Here is the description of the issue that the above commit was against: https://github.com/detro/ghostdriver/issues/247

From that you can see it was a fix to get things working with selenium-webdrivers latest protocol: "New specs: https://code.google.com/p/selenium/wiki/JsonWireProtocol#/session Change in Selenium: https://code.google.com/p/selenium/source/detail?r=8c3c65772d57"

It also looks like Updating to the latest selenium-webdriver-standalone jar will allow you to get 1.9.2 working. I have confirmed that my ruby webdriver can now connect successfully with the 2.35.0 jar and phantomjs 1.9.2.

You can get the latest jar here:

https://code.google.com/p/selenium/downloads/detail?name=selenium-server-standalone-2.35.0.jar&can=2&q=