Gitlab-CI how to use artifacts in different pipeline

Question

Currently, I have two main project.

1-) Vue Project which contains (webviews for IOS and Android, websites, and renderer for our Electron ) they are sharing components & API's.

2-) Electron Project which builds desktop app for (windows, darwin, linux)

i would like to automate our building, releasing process. my current setup..

            before_script: 
                - apt-get update
                - apt-get install zip unzip 
                - rm -rf vue-project
                - git clone vue-project
                - cd vue-project
                - git checkout dev
                - git pull
                - sed -i "/\b\(areaCode\|inline-svg-loader\)\b/d" ./packages/devtool/package.json
                - yarn install
                - ln -s vue-project/packages/desktop/ web
                - npm install

            build_darwin:
                stage: build
                script:
                    - npm run package -- darwin --deploy
                cache:
                    paths:
                        - vue-project/node_modules
                        - node_modules

which basically before bundling electron project it's cloning vue-project install dependencies and bundling electron-renderer then when it's finish. i'm running package.

I would like to separate this two different job from each other. is there anyway i could use artifacts from different project gitlab-CI pipelines ?

any help would be an appreciated.

Answer 1

Gitlab has a API for do a lot of tricks.

curl --header "PRIVATE-TOKEN:YOURPRIVATETOKEN" "https://gitlab.example.com/api/v4/projects/1/jobs/artifacts/master/download?job=test"

for download it as a file.

curl --header "PRIVATE-TOKEN:YOURPRIVATETOKEN" -o artifacts.zip "http://gitlab.example.net/api/v4/projects/<projectnumber>/jobs/artifacts/master/download?job=build_desktop

Answer 2

Gitlab can certainly support this. To accomplish this, follow these steps:

ARTIFACT GENERATION

In your Vue Project, modify your job(s) of interest to store artifacts relevant to the Electron project. Each job's artifacts are defined using Gitlab Job Artifacts notation, and are uploaded at job completion to Gitlab, and stored associated to your Project, Branch, and Job.

Note: Branch is often overlooked, and matters when you want to retrieve your artifacts, more on this later.

Illustrating:

Vue Project .gitlab_ci.yml

stages:
- stage1
- ...

vue-job1:
  stage: stage1
  script:
    - echo "vue-job1 artifact1" > ./artifact1
    - echo "vue-job1 artifact2" > ./artifact2
  artifacts:
    when: always
    paths:
    - ./artifact1
    - ./artifact2
    expire_in: 90 days

vue-job2:
  stage: stage1
  script:
    # error, would overwrite job1's artifacts since working
    # directory is a global space shared by all pipeline jobs
    # - echo "vue-job2 artifact1" > ./artifact1
    - echo "vue-job2 artifact1" > ./artifact3
  artifacts:
    when: always
    paths:
    - ./artifact3
    expire_in: 90 days

The artifacts generated above are written to the working directory, which is a clone of your project's repo. So be careful with filename conflicts. To be safe, put your artifacts in a subdirectory (eg: cat "foo" > ./subdir/artifact) and reference them in paths the same way (paths: - ./subdir/artifact). You can use 'ls' in your script to view the working directory.

When your job completes, you can confirm the artifacts stored in Gitlb by using the Gitlab UI. View the job output, and use the Browse button under Job Artifacts on the right panel.

ARTIFACT RETRIEVAL

In your Electron Project, modify your job(s) of interest to retrieve artifacts stored in the Vue Project using the Gitlab Job Artifacts API and curl. In order to access the Vue artifacts, you will need the Vue Project, Branch, and Job that the artifacts were created under.

Project: For Project, use the Project ID displayed in the Gitlab UI Project Details screen.

Branch: Usually master, but depends on the branch your pipeline executes against. Although this is not relevant for your problem, if you are generating and consuming artifacts across executions of the same pipeline, use the Gitlab variable $CI_COMMIT_BRANCH for the branch.

Job: Generally the Job Name that generated the artifacts for your Project. But if you need artifacts produced by a specific Job, then use the Job Number and the corresponding retrieval API.

Illustrating:

Electron Project .gitlab_ci.yml

stages:
- stage1
- ...

electron-job1:
  stage: stage1
  script:
  - curl -o ./artifact1 -H "PRIVATE-TOKEN:$TOKEN" https://gitlab.example.com/api/v4/projects/$VUE_PROJECT_ID/jobs/artifacts/$BRANCH/raw/artifact1?job=vue-job1
  - curl -o ./artifact2 -H "PRIVATE-TOKEN:$TOKEN" https://gitlab.example.com/api/v4/projects/$VUE_PROJECT_ID/jobs/artifacts/$BRANCH/raw/artifact2?job=vue-job1
  - curl -o ./artifact3 -H "PRIVATE-TOKEN:$TOKEN" https://gitlab.example.com/api/v4/projects/$VUE_PROJECT_ID/jobs/artifacts/$BRANCH/raw/artifact3?job=vue-job2

This script retrieves artifacts individually to the working directory of your Electron Project. There are also options for retrieving all artifacts for your job at once as a zip archive.

MISCELLANEOUS

Although this is not in the problem posed, it is worth noting that you can use artifacts both within the lifespan of a single pipeline execution to pass information between jobs. You can also use this to pass information across pipeline executions within the same project.