Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Dynamically defining parallel steps in declarative jenkins pipeline

I try to parallelize dynamically defined set of functions as follows:

def somefunc() {
    echo 'echo1'
}

def somefunc2() {
    echo 'echo2'
}

running_set = [
    { somefunc() },
    { somefunc2() }
]

pipeline {
    agent none
    stages{
        stage('Run') {
            steps {
                parallel(running_set)                
            }

        }
    }
}

And what I end up with is:

org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
WorkflowScript: 17: No "steps" or "parallel" to execute within stage "Run" @ line 17, column 9.
           stage('Run') {

Although steps are defined within stage 'Run'. Anyway what I would like to achieve running is a dynamically defined set of functions to execute in parallel.

like image 975
pbn Avatar asked Apr 10 '18 12:04

pbn


People also ask

Can we have multiple steps in a stage in Jenkins?

Jenkins Pipeline allows you to compose multiple steps in an easy way that can help you model any sort of automation process. Think of a "step" like a single command which performs a single action. When a step succeeds it moves onto the next step. When a step fails to execute correctly the Pipeline will fail.

Can parallel development be supported by Jenkins?

Jenkins parallel builds can be based on static information and decisions, in which case the declarative approach works well. But when there is a need for dynamic decisions, a new approach is required for a more advanced way of doing Jenkins parallel builds.

Which is better Scripted or declarative pipeline?

Basically, declarative and scripted pipelines differ in terms of the programmatic approach. One uses a declarative programming model and the second uses an imperative programming mode. Declarative pipelines break down stages into multiple steps, while in scripted pipelines there is no need for this.


2 Answers

If you want to use dynamic parallel block with declarative pipeline script, you have to apply two changes to your Jenkinsfile:

  1. You have to define running_set as a Map like ["task 1": { somefunc()}, "task 2": { somefunc2() }] - keys from this map are used as parallel stages names
  2. You have to pass running_set to parallel method inside script {} block

Here is what updated Jenkinsfile could look like:

def somefunc() {
    echo 'echo1'
}

def somefunc2() {
    echo 'echo2'
}

running_set = [
    "task1": {
        somefunc()
    },
    "task2": {
        somefunc2()
    }
]

pipeline {
    agent none
    stages{
        stage('Run') {
            steps {
                script {
                    parallel(running_set)
                }
            }
        }
    }
}

And here is what it looks like in Blue Ocean UI:

enter image description here

like image 186
Szymon Stepniak Avatar answered Oct 18 '22 14:10

Szymon Stepniak


It is not obvious. But Szymon's way can be very straightforward.

pipeline {
    agent none
    stages{
        stage('Run') {
            steps {
                script {
                    parallel([
                        'parallelTask1_Name': {
                            any code you like
                        },
                        'parallelTask2_Name': {
                            any other code you like
                        },
                        ... etc
                    ])
                }
            }
        }
    }
}
like image 29
naZOID Avatar answered Oct 18 '22 14:10

naZOID