Cypress tests are green locally, but failing in CI - tips on debugging?

Question

As much as I adore Cypress it's starting to turn out quite badly. I don't think I would be doing something fundamentally wrong. I've read best practices a couple of times and I don't see what I could improve really.

It's starting to be rather frustrating. Having tests working perfectly on the local machine (tried running several times in the row) and yet when the same code runs through the CI (currently Bitbucket Pipelines), several tests fail for weird reasons.

For example, a single click on an item in the list adds a single item in the cart. Works perfectly however I try to break it and yet the same test in CI makes that click happen twice there for some reason. And that's one issue I am at least able to describe. Others are out of whack and very often happening just randomly like that element is no visible and yet looking at the screenshot I can see just fine.

I tried to use cypress-failed-log plugin to see command log, but that does not really help because it's the same thing I see locally and yet it fails in CI. A video I can see in Cypress Dashboard is not that helpful either because it's usually way too fast and some things are not even seen there.

Can someone advise me some other options on how to approach the issue at hand more gracefully? I must admit I am on the verge of giving up on these tests because it just takes too much time to make them reliable.

---

Some details about my setup:

[email protected]
[email protected]
[email protected]

# job definition for running E2E tests in parallel
e2e: &e2e
  name: E2E tests
  image: cypress/browsers:chrome67-ff57
  caches:
    - yarn
    - home-cache
  script:
    - yarn -v
    - cd cypress
    - yarn install --frozen-lockfile
    - npx @bahmutov/print-env BITBUCKET
    - yarn ci --parallel --ci-build-id $BITBUCKET_BUILD_NUMBER

Answer 1

Weird, do you use the same browser when you do your local test ? You use additional npm packages, maybe you could remove all the non essentials and retry ?