GitLab CI cannot find module error (when it is in node_modules)

Question

I'm trying to run som client-side tests that connect to an endpoint on a server, which I'm using chai.require("server.js") to set up. This works on my local machine, but when I try and run this on GitLab CI, it fails and says it cannot find the modules inside the server.js file, even though they exist in node_modules (I printed these, and checked for the exact one it asked for). I also moved the require-s in server.js around, and it always fails on the first one / fails on all of them.

This is the relevant parts of my package.json file

{
"name": "client",
"version": "0.1.0",
"private": true
"dependencies": {
  "axios": "^0.19.1",
  "chai": "^4.2.0",
  "chai-http": "^4.3.0",
  "esm": "^3.2.25",
  "mocha": "^7.0.0",
  "mysql": "^2.17.1"
  "react-scripts": "3.3.0",
  "body-parser": "^1.19.0",
  "cors": "^2.8.5",
  "express": "^4.17.1",
  "install": "^0.13.0",
  "jest": "^24.9.0",
  "node": "^13.5.0",
  "npm": "^6.13.4",
  "require": "^2.4.20"
},
"scripts": {
  "start": "react-scripts start",
  "build": "react-scripts build",
  "test": "mocha --require esm \"src/**/*.test.js\"",
  "eject": "react-scripts eject"
},
"eslintConfig": {
  "extends": "react-app"
},
"browserslist": {
  "production": [
  ">0.2%",
  "not dead",
  "not op_mini all"
],
"development": [
  "last 1 chrome version",
  "last 1 firefox version",
  "last 1 safari version"
]
}, "devDependencies": {
  "react-router-dom": "^5.1.2",
  "jest": "^24.9.0"
}
}

And my .gitlab-ci.yml:

image: node:13

# Set up MySQL test-database
services:
  - mysql:5.5

# Configure test-database
variables:
  MYSQL_DATABASE: supertestdb
  MYSQL_ROOT_PASSWORD: secret

# Install node libraries
before_script:
  - cd application/client
  - npm install

# Run JEST test

run_tests:
  stage: test
  script:
    - npm test

The error stacktrace from the CI Job:

[email protected] test /builds/williaj/harmoni-team-7/application/client
62 > mocha --require esm "src/**/*.test.js"
63 /builds/williaj/harmoni-team-7/application/server/src/server.js:1
64 Error: Cannot find module 'express'
65 Require stack:
66 - /builds/williaj/harmoni-team-7/application/server/src/server.js
67 - /builds/williaj/harmoni-team- 
      7/application/client/src/test/storeTests/cookies.test.js
68 - /builds/williaj/harmoni-team- 
      7/application/client/node_modules/mocha/lib/mocha.js
69 - /builds/williaj/harmoni-team- 
      7/application/client/node_modules/mocha/lib/cli/one-and-dones.js
70 - /builds/williaj/harmoni-team- 
      7/application/client/node_modules/mocha/lib/cli/options.js
71 - /builds/williaj/harmoni-team- 
      7/application/client/node_modules/mocha/lib/cli/cli.js
72     at Object.<anonymous> (/builds/williaj/harmoni-team- 
      7/application/server/src/server.js:1)
73     at Generator.next (<anonymous>)
74     at Object.Module._extensions..js 
       (internal/modules/cjs/loader.js:1164:10)
75     at Object.<anonymous> (/builds/williaj/harmoni-team- 
      7/application/client/src/test/storeTests/cookies.test.js:12:13)
76     at Generator.next (<anonymous>)
77     at Object.Module._extensions..js 
      (internal/modules/cjs/loader.js:1164:10)
78     at /builds/williaj/harmoni-team- 
      7/application/client/node_modules/mocha/lib/mocha.js:308:36
79     at Array.forEach (<anonymous>)
80     at Mocha.loadFiles (/builds/williaj/harmoni-team- 
      7/application/client/node_modules/mocha/lib/mocha.js:305:14)
81     at Mocha.run (/builds/williaj/harmoni-team- 
      7/application/client/node_modules/mocha/lib/mocha.js:846:10)
82     at Object.exports.singleRun (/builds/williaj/harmoni-team- 
      7/application/client/node_modules/mocha/lib/cli/run-helpers.js:108:16)
83     at exports.runMocha (/builds/williaj/harmoni-team- 
      7/application/client/node_modules/mocha/lib/cli/run-helpers.js:143:13)
84     at Object.exports.handler (/builds/williaj/harmoni-team- 
      7/application/client/node_modules/mocha/lib/cli/run.js:305:3)
85     at Object.runCommand (/builds/williaj/harmoni-team- 
      7/application/client/node_modules/yargs/lib/command.js:242:26)
86     at Object.parseArgs [as _parseArgs] (/builds/williaj/harmoni-team- 
      7/application/client/node_modules/yargs/yargs.js:1087:28)
87     at Object.parse (/builds/williaj/harmoni-team- 
      7/application/client/node_modules/yargs/yargs.js:566:25)
88     at Object.exports.main (/builds/williaj/harmoni-team- 
      7/application/client/node_modules/mocha/lib/cli/cli.js:68:6)
89     at Object.<anonymous> (/builds/williaj/harmoni-team- 
      7/application/client/node_modules/mocha/lib/cli/cli.js:73:11)
90     at Generator.next (<anonymous>)
91     at internal/main/run_main_module.js:17:47 {
92   code: 'MODULE_NOT_FOUND',
93   requireStack: [
94     '/builds/williaj/harmoni-team-7/application/server/src/server.js',
95     '/builds/williaj/harmoni-team- 
     7/application/client/src/test/storeTests/cookies.test.js',
96     '/builds/williaj/harmoni-team- 
     7/application/client/node_modules/mocha/lib/mocha.js',
97     '/builds/williaj/harmoni-team- 
     7/application/client/node_modules/mocha/lib/cli/one-and-dones.js',
98     '/builds/williaj/harmoni-team- 
     7/application/client/node_modules/mocha/lib/cli/options.js',
99     '/builds/williaj/harmoni-team- 
     7/application/client/node_modules/mocha/lib/cli/cli.js'
100   ]
101 }
102 npm ERR! Test failed.  See above for more details.

And to re-iterate, it's the chai.require('server.js') that then again run the imports that crash the CI, and this runs on my machine. Thankful for any answers.

Answer 1

Happened something similar to me with pipeline cache, in my case what solved the problem appears to be "Clear Runner caches" through the Gitlab UI.

Read this documentation in order to understand about what button i'm talking about

Answer 2

If this could be relevant, I had a similar issue on my pipeline. I moved the cache config from a job to the global, leading to this conf :

    image: weboaks/node-karma-protractor-chrome
    
    cache:
        paths:
        - node_modules/
    
    stages:
      - test
      - release
      - postrelease
      - deploy
      - e2e

I no longer had the issue when running npm command from multiple jobs.