Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Cypress: interrupt all tests on first failure

How to interrupt all Cypress tests on the first test failure?

We are using semaphore to launch complete e2e tests with Cypress for each PR. But it takes too much time.

I'd like to interrupt all tests on the first test failure.

Getting the complete errors is each developer's business when they develop. I just want to be informed ASAP if there is anything wrong prior to deploy, and don't have to wait for the full tests to complete.

So far the only solution I came up with was interrupting the tests on the current spec file with Cypress.

afterEach(() => {
        if (this.currentTest.state === 'failed') {
            Cypress.runner.end();
        }
    });

But this is not enough since it only interrupts the tests located on the spec file, not ALL the other files. I've done some intensive search on this matter today and it doesn't seem like this is a thing on Cypress.

So I'm trying other solutions.

1: with Semaphore

fail_fast:
  stop:
    when: "true"

It is supposed to interrupt the script on error. But it doesn't work: tests keep running after error. My guess is that Cypress will throw an error only when all tests are complete.

2: maybe with the script launching Cypress, but I'm out of ideas Right now here are my scripts

"cy:run": "npx cypress run",
"cy:run:dev": "CYPRESS_env=dev npx cypress run",
"cy:test": "start-server-and-test start http-get://localhost:4202 cy:run"
like image 305
Florent Arlandis Avatar asked May 07 '20 15:05

Florent Arlandis


People also ask

How do you skip failed test cases in Cypress?

skip in order to skip a test in a cypress suite.

How can we stop testing in Cypress?

Cypress gives us the ability to stop the test at a spot via cy. pause() command. We can use this to stop the test before any action or assertion that is causing our tests to fail. But this command only works when we run Cypress in GUI mode and, it is ignored when we run the tests in headless mode.

Is Cypress good for e2e testing?

By using the Cypress framework, testers can easily achieve the e2e coverage for a given web application by using the TDD method. Unlike Selenium, as Cypress is not dependent on browser drivers, since it directly interacts with the Browsers.

How do you filter tests in Cypress?

Tests may be filtered through the Cypress Module API by creating a JavaScript file containing Cypress run parameters. As an example, we will create a module which runs all tests within the Elements page module.


3 Answers

EDIT: It seems like this feature was introduced, but it requires paid version of Cypress (Business Plan). More about it: Docs, comment in the thread


Original answer:

This has been a long-requested feature in Cypress for some reason still has not been introduced. There are some workarounds proposed by the community, however it is not guaranteed they will work. Check this thread on Cypress' Github for more details, maybe you will find a workaround that works for your case.

like image 89
P D Avatar answered Oct 07 '22 22:10

P D


A little hack that worked for me

Cypress.Commands.add('interrupt', () => {
  eval("window.top.document.body.querySelector('header button.stop').click()");
});
like image 5
user3504541 Avatar answered Oct 08 '22 00:10

user3504541


The solution by @user3504541 is excellent! Thanks a ton. I already started giving up on using Cypress since these issues keep popping up. But in any case, here's my config:

support/index.ts

declare global {
  // eslint-disable-next-line
  namespace Cypress {
    interface Chainable {
      interrupt: () => void
    }
  }
}

function abortEarly() {
  if (this.currentTest.state === 'failed') {
    return cy.task('shouldSkip', true)
  }
  cy.task('shouldSkip').then(value => {
    if (value) return cy.interrupt()
  })
}

commands/index.ts

Cypress.Commands.add('interrupt', () => {
  eval("window.top.document.body.querySelector('header button.stop').click()")
})

In my case the Cypress tests were left pending indefinitely on the CI (Github action workflow) but with this fix they interrupt properly.

like image 5
TeemuK Avatar answered Oct 07 '22 23:10

TeemuK