Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Sequential stages within parallel pipeline in Jenkins

I have a dynamic scripted pipeline in Jenkins that has many parallel stages, but within each stage, there are multiple serial steps. I have wasted several days trying to make it work: no matter what I try, all serial substages are lumped into one stage! Here is what I have now:

node () {
    stage("Parallel Demo") {
        // Canonical example to run steps in parallel

        // The map we'll store the steps in
        def stepsToRun = [:]

        for (int i = 1; i < 5; i++) {
            stepsToRun["Step${i}"] = { node {
                echo "start"
                sleep 1
                echo "done"
            }}
        }
        // Actually run the steps in parallel
        // parallel takes a map as an argument
        parallel stepsToRun
    }
}

It gets me this beautiful parallel pipeline:

enter image description here

However, the moment I add a serial stage, aka:

node () {
    stage("Parallel Demo") {
        // Run steps in parallel

        // The map we'll store the steps in
        def stepsToRun = [:]

        for (int i = 1; i < 5; i++) {
            stepsToRun["Step${i}"] = { node {
                stage("1") {
                    echo "start 1"
                    sleep 1
                    echo "done 1"
                }
                stage("2") {
                    echo "start 2"
                    sleep 1
                    echo "done 2"
                }                
            }}
        }
        // Actually run the steps in parallel
        // parallel takes a map as an argument
        parallel stepsToRun
    }
}

I get this ugly thing, which looks exactly the same:

enter image description here

To add to the offense, I see the sub-steps executed. How can I get my sub-steps show up as stages?

Also, if there is a way to have dynamic stages (sequential and parallel) with the declarative pipeline, I'm all for it. I found you can do static sequential stages but have little clue how to make it dynamic without going back to scripted pipelines.

like image 931
Oleksiy Avatar asked Jan 10 '20 21:01

Oleksiy


People also ask

Can we have multiple stages in Jenkins pipeline?

Pipelines are made up of multiple steps that allow you to build, test and deploy applications. Jenkins Pipeline allows you to compose multiple steps in an easy way that can help you model any sort of automation process. Think of a "step" like a single command which performs a single action.

Are Jenkins stages sequential?

What are sequential stages in Jenkins pipeline? Stages in Declarative Pipeline may have a stages section containing a list of nested stages to be run in sequential order. Note that a stage must have one and only one of steps, stages, parallel, or matrix.

What are the stages in Jenkins pipeline?

It contains a collection of states such as build, deploy, test and release. These jobs or events are interlinked with each other. Every state has its jobs, which work in a sequence called a continuous delivery pipeline.

Can Jenkins do parallel stages?

Jenkins Pipelines can do parallel stages for a while, even in the Declarative format 1. Although doing parallel pipelines, Jenkins didn't become awesome until Sequential Stages 2. We will dive into the magic of Sequential Stages, but first, let's start with building in parallel. This is an elementary example.

How can I speed up a Jenkins pipeline?

There are many ways to speed up builds, do fewer tests, get bigger and better hardware, or run some tasks in parallel. Jenkins Pipelines can do parallel stages for a while, even in the Declarative format 1. Although doing parallel pipelines, Jenkins didn't become awesome until Sequential Stages 2.

What is the use of Jenkins in declarative pipeline?

In declarative pipelines, Jenkins allows the definition of parallel stages. It further allows scripted pipeline general purpose scripts to create and manipulate the artifacts of the declarative pipeline.

What is the difference between parallel stages and sequential stages?

With Parallel Stages, we can execute some steps in parallel, but with regards to visualizing individual steps, it is inferior. Sequential Stages allows us to add Stages in sequence within a Parallel step, which is why I usually call them Parallel Sequential stages.


1 Answers

Here is how you can do something like you want

def stepsToRun = [:]

pipeline {
    agent none

    stages {
        stage ("Prepare Stages"){
            steps {
                script {
                    for (int i = 1; i < 5; i++) {
                        stepsToRun["Step${i}"] = prepareStage("Step${i}")
                    }   
                    parallel stepsToRun
                }
            }
        }
    }
}

def prepareStage(def name) {
    return {
        stage (name) {
            stage("1") {
                echo "start 1"
                sleep 1
                echo "done 1"
            }
            stage("2") {
                echo "start 2"
                sleep 1
                echo "done 2"
            }
        }
    }
}

enter image description here

like image 120
Sers Avatar answered Oct 21 '22 04:10

Sers