Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Share gitlab-ci.yml between projects

We are thinking to move our ci from jenkins to gitlab. We have several projects that have the same build workflow. Right now we use a shared library where the pipelines are defined and the jenkinsfile inside the project only calls a method defined in the shared library defining the actual pipeline. So changes only have to be made at a single point affecting several projects.

I am wondering if the same is possible with gitlab ci? As far as i have found out it is not possible to define the gitlab-ci.yml outside the repository. Is there another way to define a pipeline and share this config with several projects to simplify maintainance?

like image 654
pyriand3r Avatar asked Dec 13 '17 10:12

pyriand3r


People also ask

Can GitLab CI Yml have multiple files?

No, you can't have multiple gitlab-ci files per repository.

Where should the GitLab CI Yml file be placed?

gitlab-ci. yml in the root of your repository, which contains the CI/CD configuration.

What are shared runners in GitLab?

A runner is an isolated (virtual) machine that picks up builds through the coordinator API of GitLab CI. A runner can be specific to a certain project or serve any project in GitLab CI. A runner that serves all projects is called a shared runner.

Can you have multiple pipelines in GitLab?

You can set up GitLab CI/CD across multiple projects, so that a pipeline in one project can trigger a pipeline in another project. You can visualize the entire pipeline in one place, including all cross-project interdependencies.


2 Answers

First let me start by saying: Thank you for asking this question! It triggered me to search for a solution (again) after often wondering if this was even possible myself. We also have like 20 - 30 projects that are quite identical and have .gitlab-ci.yml files of about 400 - 500 loc that have to each be changed if one thing changes.

So I found a working solution:

Inspired by the Auto DevOps .gitlab-ci.yml template Gitlab itself created, and where they use one template job to define all functions used and call every before_script to load them, I came up with the following setup.

  • Multiple project repo's (project-1, project-2) requiring a shared set of CI jobs / functions
  • Functions script containing all shared functions in separate repo

Files

So using a shared ci jobs scipt:

#!/bin/bash  function list_files {   ls -lah }  function current_job_info {   echo "Running job $CI_JOB_ID on runner $CI_RUNNER_ID ($CI_RUNNER_DESCRIPTION) for pipeline $CI_PIPELINE_ID" } 

A common and generic .gitlab-ci.yml:

image: ubuntu:latest  before_script:   # Install curl   - apt-get update -qqq && apt-get install -qqqy curl   # Get shared functions script   - curl -s -o functions.sh https://gitlab.com/giix/demo-shared-ci-functions/raw/master/functions.sh   # Set permissions   - chmod +x functions.sh   # Run script and load functions   - . ./functions.sh  job1:   script:     - current_job_info     - list_files 

You could copy-paste your file from project-1 to project-2 and it would be using the same shared Gitlab CI functions.

These examples are pretty verbose for example purposes, optimize them any way you like.

Lessons learned

So after applying the construction above on a large scale (40+ projects) I want to share some lessons learned so you don't have to find out the hard way:

  • Version (tag / release) your shared ci functions script. Changing one thing can now make all pipelines fail.
  • Using different Docker images could cause an issue in the requirement for bash to load the functions (e.g. I use some Alpine-based images for CLI tool based jobs that have sh by default)
  • Use project based CI/CD secret variables to personalize build jobs for projects. Like environment URL's etc.
like image 141
Stefan van Gastel Avatar answered Sep 23 '22 23:09

Stefan van Gastel


GitLab 11.7 introduces new include methods, such as include:file: https://docs.gitlab.com/ee/ci/yaml/#includefile

include:   - project: 'my-group/my-project'     ref: master     file: '/templates/.gitlab-ci-template.yml' 

This will allow you to create a new project on the same GitLab instance which contains a shared .gitlab-ci.yml.

like image 28
miq Avatar answered Sep 21 '22 23:09

miq