Gitlab CI pipeline to run jobs parallel in same stage and invoke/trigger other jobs of same stage

Question

I am trying to create a automation pipeline for data load. I have a scenario as explained below:

stages
 - stage1
 - stage2

job1:
  stage: stage1
  script:
  - echo "stage 1 job 1"


job2:
  stage: stage1
  script:
   - echo "stage 1 job 2"

job3:
  stage: stage1
  script:
   - echo "stage 1 job 3"

job4:
  stage: stage1
  script:
   - echo "stage 1 job 4"

I want to run the job1 and job2 parallel in the same stage. So, after Job1 and job2 success

• job1 will invoke/trigger the job3. that means job3 will start automatically when job1 successes
• job2 will invoke/trigger the job4 that means job4 will start automatically when job2 successes

I am writing the pipeline in the .gitlab-ci.yml.

Can anyone help me to implement this?

Answer 1

Strict implementation of your requirements is not possible (according to my knowledge), the jobs 3 and 4 would need to be in a separate stage (although support for putting them in the same stage is planned). To be clear: the other functional requirements can be fulfilled, i.e:

• job1 and job2 start in parallel
• job1 will trigger the job3 (immediately, without waiting for the job2 to finish)
• job2 will trigger the job4 (immediately, without waiting for the job1 to finish)

The key is using needs keyword to convert the pipieline to a directed acyclic graph:

stages:
    - stage-1
    - stage-2

job-1:
    stage: stage-1
    needs: []
    script: 
      - echo "job-1 started"
      - sleep 5
      - echo "job-1 done"

job-2:
    stage: stage-1
    needs: []
    script: 
      - echo "job-2 started"
      - sleep 60
      - echo "job-2 done"

job-3:
    stage: stage-2
    needs: [job-1]
    script: 
      - echo "job-3 started"
      - sleep 5
      - echo "job-3 done"

job-4:
    stage: stage-2
    needs: [job-2]
    script: 
      - echo "job-4 started"
      - sleep 5
      - echo "job-4 done"

As you can see on the screenshot, the job 3 is started, even though the job 2 is still running.