How to install PhantomJS for use with Python Selenium on the Raspberry Pi?

Question

I want to run a Python script using Selenium WebDriver with PhantomJS as a headless browser on my Raspberry Pi running Raspbian.

I originally wrote the script in OS X where it works fine. But in trying to make it work on the Raspberry, I'm having problems.

When trying to run the script, I get this error:

raise WebDriverException("Can not connect to the Service %s" % self.path)
selenium.common.exceptions.WebDriverException: Message: Can not connect to the Service /usr/bin/phantomjs

Brief version of the script:

from selenium import webdriver
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities

user_agent = ("Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_4) " +
    "AppleWebKit/537.36 (KHTML, like Gecko) Chrome/29.0.1547.57 Safari/537.36")

dcap = dict(DesiredCapabilities.PHANTOMJS)
dcap["phantomjs.page.settings.userAgent"] = user_agent

serv_args = ["--ignore-ssl-errors=false", "--ssl-protocol=tlsv1", 
    "--disk-cache=false"]

driver = webdriver.PhantomJS(executable_path="/usr/bin/phantomjs", 
    desired_capabilities = dcap, service_arguments = serv_args, port=65000)

I have seen other people have problems resembling mine – with varying solutions – most seem to involve building PhantomJS yourself, or clone and install a Github branch adapted for Raspberry (which now is out-of-sync with the main PhantomJS project).

Questions

• Does anyone know how to resolve the problem – and indeed what the problem really is about?
• If the solution involves installing the binaries manually to /usr/local/bin or such, how would I do that? The binaries available on the PhantomJS webpage is for linux-x86 and linux-i686, so I'm assuming they won't work on an Raspberry Pi 2 B ARM Cortex A-7 processor.
• I have also tried to build PhantomJS myself according to these instructions, but the process froze midway through. The Raspberry also doesn't meet the recommended hardware requirements for building.

Background info

• I am using Python 2.7.9
• I have created a virtualenv and installed all the Python modules in it; e.g. pip install selenium, and attempt to run the script here
• I have installed the latest version of PhantomJS via sudo apt-get install phantomjs
• I have disabled my ufw firewall while testing

Answer 1

Ok I will start with the solution, there is a version compiled for arm here phantomjs-linux-armv6l, on the pi run the following commands:

$ cd /tmp
$ wget https://github.com/aeberhardo/phantomjs-linux-armv6l/archive/master.zip
$ unzip master.zip
$ cd phantomjs-linux-armv6l-master
$ bunzip2 *.bz2 && tar xf *.tar

I added:

sudo cp phantomjs-1.9.0-linux-armv6l/bin/phantomjs  /usr/bin

So phantomjs will be on your path.

pi@raspberrypi ~ $ phantomjs --version
1.9.0

pi@raspberrypi ~ $ phantomjs
phantomjs> 

Now we have that done, time to test:

pi@raspberrypi ~ $ cat test.py
#!/usr/bin/python
from selenium import webdriver

driver = webdriver.PhantomJS()
driver.get('http://stackoverflow.com/questions/36314771/how-to-install-phantomjs-for-use-with-python-selenium-on-the-raspberry-pi/36388824#36388824')
a = driver.find_element_by_xpath('//*[@id="question-header"]/h1/a')
print(a.text)
print(driver)
pi@raspberrypi ~ $ python test.py 
How to install PhantomJS for use with Python Selenium on the Raspberry Pi?
<selenium.webdriver.phantomjs.webdriver.WebDriver (session="b184e110-f9c4-11e5-aede-7f5c42f062d7")>

From the faq. Starting with PhantomJS 1.5, it is pure headless and there is no need to run X11/Xvfb anymore..

I tried using xvfb-run and exporting the display, using a shell script in init.d to start xvfb, I got a little further being able to run iceweasel from bash headless no problem but still no cigar when it came to phantomjs and selenium. I think it may just come down to an incompatibility between selenium and the version of phantomjs, regardless having 1.9.0 and real headless browsing is a lot more desirable.

I was in the middle of setting up a toolchain and was going to try to compile myself when I found the link above, for anyone interested in cross compiling, crosstools-ng makes life a lot easier.

I am running an arm6, there is also a compiled version for arm7 using 2.0.0, the dependencies are:

sudo apt-get install flex bison gperf ruby perl libsqlite3-dev libfontconfig1-dev libicu-dev libfreetype6 libssl-dev libpng-dev libjpeg-dev python libX11-dev libxext-dev

The install procedure, I have extracted the binary to dropbox:

wget https://www.dropbox.com/s/epj1rji9d239dco/phantomjs
chmod +x phantomjs
sudo cp phantomjs /usr/bin

The original github link is phantomjs-2.0.0-armv7