Protractor Accessibility reporting

Question

I am trying to use Accessibility plugin that comes with Protractor. From what I see it does checking for a11y of last page that I am located. Is there a way to have 2 test scripts executed one after another one and provide different reports or put all in one report but separated.

Example:

access.js

access1.js

Output file:

resultJsonOutputFile: 'result/result.json'

I tried to this way in conf.js:

specs: ['../test/access.js', '../test/access1.js'],
or
specs: ['../test/access*.js'],

but still get result for last script executed

I tried also creating suites:

suites: {
   homepage: '../test/homepage/access.js',
   catalogpage: '../test/catalogpage/access1.js'
  },

but when I check JSON file, if 2 scripts executed, then 1st one is ok with no issues and provides error for 2nd script. However, if to run 1st script alone, Protractor provides errors

Also I tried to create in one js file as different scenarios, but still same issue

Answer 1

With the current implementation, the accessibility plugin is set to run exactly once per invocation of the Protractor runner, on the last page. So unfortunately, no modification of the suites or test files will make it run more than once.

You can create separate configuration files for each set of test files you'd like to run, or using shardTestFiles to make sure that each file is run in its own process. See the referenceConf for more details on sharding.

Answer 2

Alternatively, you could use aXe to do your accessibility testing. In order to use it with e2e tests in protractor and Webdriver, do the following:

npm install --save-dev axe-webdriverjs

Then in your e2e test files, you do:

var AxeBuilder = require('path_to_the/axe-webdriverjs');

to get hold of the AxeBuilder and then wherever you need to run a test, you:

AxeBuilder(browser.driver)
    .analyze(function (results) {
        expect(results.violations.length).toBe(0);
    });

The above example is using Jasmine but you can extrapolate for any other assertion library.

Also: there is a sample project you can clone and run here https://github.com/dylanb/UITestingFramework

Disclaimer: I am associated with the aXe project and therefore not neutral