Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to share cache between runners that run concurrently in the same stage

Tags:

gitlab-ci

I'm totally new to gitlab CI and I'm playing around with pipeline and trying to create one from scratch for webapp.

I have a cache that is created on first stage of the pipeline (typical npm cache based on yarn.lock).

  cache:
    key:
      files:
        - yarn.lock
    paths:
      - node_modules
  only:
    refs:
      - merge_requests
      - master
    changes:
      - yarn.lock

On second stage, I run build & test tasks concurrently.

When only one task is on second stage (either build or test), the the same runner runner-4qsydtwu-project-2077-concurrent-0 is used and cache is reused.

However, when both build and test tasks are executed concurrently, then runner-4qsydtwu-project-2077-concurrent-1 is used for second task, and cache is not available.

How can I make runner *-1 reuse the cache?

like image 427
dragonfly Avatar asked Oct 24 '25 17:10

dragonfly


1 Answers

Taken from Caching in GitLab CI/CD: Good caching practices:

For runners to work with caches efficiently, you must do one of the following:

  • Use a single runner for all your jobs.
  • Use multiple runners that have distributed caching, where the cache is stored in S3 buckets. Shared runners on GitLab.com behave this way. These runners can be in autoscale mode, but they don’t have to be.
  • Use multiple runners with the same architecture and have these runners share a common network-mounted directory to store the cache. This directory should use NFS or something similar. These runners must be in autoscale mode.

So if you want to use multiple runners you can choose between options 2 and 3.

like image 63
slauth Avatar answered Oct 27 '25 00:10

slauth