Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

GitLab CI job takes 20+ minutes

With gitlab-ci I am using a simple .yml file. I have defined various stages to run synchronously. I have set a cache for node_modules. But the problem is that the cache of node_modules is actually slowing down the process. This cache is required to make the node_modules the same across each stage. (Each stage automatically clears /node_modules for some reason)

When building locally this whole process takes less then 2 minutes. But on the CI machine this process takes between 20 and 25 minutes. Learning how Gitlab CI works internally, I've learned that it's zipping the node_module files (about 36K small files) and that process is extremely slow.

tl;dr: What is the proper way to handle node_module caching with Gitlab CI without uploading node_modules to artifacts? I would like to avoid uploading artifacts that are over 400MB large.

See configuration below:

cache:
  untracked: true
  key: "%CI_COMMIT_REF_NAME%"
  paths:
    - node_modules

stages:
  - install
  - eslint-check
  - eslint
  - prettier
  - test
  - dist

# install dependancies
install:
  stage: install
  script:
    - yarn install
  environment:
    name: development

# run eslint-check
eslint-check:
  stage: eslint-check
  script:
    - yarn eslint-check
  environment:
    name: development

# Other scripts below
like image 922
Perfection Avatar asked Nov 08 '22 04:11

Perfection


1 Answers

It would appear that there will be a solution for this in the future as the issue has been discussed here for almost two years. A milestone has been set so this can be resolved eventually.

https://gitlab.com/gitlab-org/gitlab-runner/issues/1797

like image 57
Perfection Avatar answered Nov 13 '22 23:11

Perfection