How do you write end-to-end tests for Polymer (JS) based application (circa May 2015)?

Question

I have built a Polymer based application. I'd like to write some end-to-end tests (not unit tests, but user behavior integration tests) for it. How do I do this currently (May 2015)?

Answer 1

I spent the past few days looking into this problem. Despite the vast number of pages devoted to one related topic or another on the web, nothing documents a solution to this problem. I was able to piece together something that works. So here it is. Hope this is useful for those looking to write end-to-end tests for Polymer apps. I am sure a better solution will come about as Polymer, web components, and shadow DOM technologies mature.

Some details: Linux, want automate testing in a script (ideally headless), app consists of many Polymer elements and is served by and loads data from a Django server.

Failed attempt with PhantomJS

First, I tried using casperjs and phantomjs. phantomjs is headless and casperjs has very good navigation support, so I thought those would be a good combination to end up with. Unfortunately, phantomjs does not support HTML5 imports and the webcomponents.js polyfill does not seem to work on phantomjs.

Using Selenium

I ended up with a Selenium/ChromeDriver based solution, using the selenium python client. I did not test this with Firefox driver so I don't know if that works. Here is what you need to do:

To make things easier, create a directory to put stuff in:

mkdir selenium

Install google chrome. I did this via the Google website and downloaded the linux version. Then, download selenium server 2.45 and chromedriver 2.15, into the selenium directory. E.g.

$ ls selenium/
chromedriver  selenium-server-standalone-2.45.0.jar

Then, install Python Selenium API

$ pip install selenium

Run a simple test:

$ cd selenium
$ cat > test.py
from selenium import webdriver
driver = webdriver.Chrome('./chromedriver')
driver.get("http://localhost:8000/myapp/")
driver.implicitly_wait(3)
print driver.title
content = driver.find_element_by_css_selector('myelement::shadow h3')
print content.text
driver.close()
$ xvfb-run python test.py

(xvfb is necessary to make the running of test.py headless)

test.py prints out the content of the h3 element in a custom Polymer element called myelement. E.g. if DOM looks like

<myelement>
  <h3>Hello World</h3>
</myelement>

Then test.py prints "Hello World".

The h3 element appears in the shadow DOM of myelement. Chrome dev tool lists the CSS selector for this h3 as "myelement #shadow-root h3". Using Selenium, you can access this h3 using "myelement::shadow h3" as the CSS selector.

Tests and Test Data

I organized my tests as Python unittest test cases, and wrote a test driver script. The driver script forks, creates a Django dev server in the child process, and runs "python -m unittest" in the parent process. Each test case uses Python selenium API to connect to the Django dev server. In the setUp and tearDown methods of each test case, I inject test data into the database using my Django model classes.

I run the driver script under xvfb -- "xvfb-run python driver.py" -- to make it headless.

Dealing with Ajax and two-way bindings

My Polymer app uses ajax to load data and two-way bindings to render HTML templates. Polymer also renders templates and updates the DOM asynchronously. In my test cases, I depend on Selenium's conditional wait to detect when data loading completes and DOM re-renders. Implicit waiting (which isn't a good idea anyways) did not work for me for some reason; the implicit wait just returns immediately. I also updated my HTML templates to be more testable -- adding DOM IDs and unique text or CSS selectors to differentiate different stages of the app.

Caveats

A button with only as its inner HTML becomes un-clickable using Selenium. If you have such a button, use ActionChains to click it:

chain = ActionChains()
chain.move_to_element(element)
chain.click()
chain.perform()