E2E Test Metrics

Question

Apologies if this question is better suited for Stack Exchange Programmers, I've posted this on both sites cause I figured it was on the fence.

Question: Are there any valuable metrics on E2E automated tests? (Does it make sense to gather any data around them?)

Context: For example, when I wrote my unit tests, I implemented a test coverage package which covers % of Classes, # of methods touched etc...

A few points I've come across:

• Test efficiency (speed of specs)
• App performance (response/loading time)
• Automation Progress (# of tests currently automated VS # of tests able to be automated)
• Defect Efficiency (# of defects found during testing VS # of defects found after delivery)

Any ideas? If it matters, I'm using Protractor on an Angular app.

Answer 1

Something that comes to my mind with protractor: You can measure code coverage with e2e tests as well, but that would require some tricks to setup, check this: https://www.npmjs.com/package/protractor-istanbul-plugin https://www.npmjs.com/package/grunt-protractor-coverage

Be aware that it is not clean solution - your code could be minified, server side not included, just keep this in mind.

Also you could measure page performance (load speed, JS execution speed, CSS render other client-side calculations) with something like protractor-perf: https://github.com/axemclion/protractor-perf

But also keep in mind that it requires a lot of preparations to achieve that, but anyway it is cool.

About Automation Progress/Percent automatable - i think you can't track this automatically, only if your requirements will be very detailed, and stored in some system with API. Then you could link your test case with specific requirement and track that. I never saw this working actually.

Defect Efficiency - easier to track with JIRA reports.