Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Jenkins Pipeline "node inside stage" vs "stage inside node"

As both node step and stage step provide scoped {} syntax, what is the best practice for defining their topology inside groovy code?

Exhibit A

node ("NodeName") {
    stage ("a stage inside node"){
        // do stuff here
    }
}

Exhibit B

stage ("a stage holding a node") {
    node ("NodeName"){
        // do stuff here
    }
}
like image 843
Rakib Avatar asked Jan 15 '17 11:01

Rakib


People also ask

Can we use different nodes for each stage in Jenkins?

You can also mix and match node { stage {..}} and stage { node {..}} within your pipeline codes. By design (in Jenkins, as well as in the concept of continuous delivery), stages do not execute in parallel. Only the steps within a single stage are executed in parallel.

What is node and stage in Jenkins pipeline?

node specifies where something shall happen. You give a name or a label, and Jenkins runs the block there. stage structures your script into a high-level sequence. Stages show up as columns in the Pipeline Stage view with average stage times and colours for the stage result.

Can we have stage inside stage in Jenkins?

Support for nested and parallel stages in Jenkins pipelinesYou can use nested and parallel stages in scripted Jenkins pipelines to automate and speed up tasks that can be run in parallel.

What is stage in Jenkins pipeline?

Stage. A stage block defines a conceptually distinct subset of tasks performed through the entire Pipeline (e.g. "Build", "Test" and "Deploy" stages), which is used by many plugins to visualize or present Jenkins Pipeline status/progress.


2 Answers

This depends on your actual needs.

As long as you can run your complete pipeline on a single node, I would wrap the stages in a node so that the pipeline is not blocked by busy executors.

As soon as you use the parallel step, then you don't really have a choice besides having stage around node allocations.

There are (at least for me) no issues around mixing that, i.e., have the first 2-3 stages executed on the same node and then one stage that executes on multiple nodes within parallel.

like image 92
StephenKing Avatar answered Oct 05 '22 19:10

StephenKing


With node { stage { ... } } each stage will share the same working folder and all the files from the previous stage will be there for the next stage.

With stage { node { ... } } you need to stash/unstash files between each stage. If you have a large repository, and especially if you have a large folder of dependencies like node_modules, this repeated stash/unstash could end up being a significant, or even majority, or your build time.

IMO I would generally start with the first syntax, node { stage { ... } } as preferred. If you have individual build stages that take time and can benefit from parallelism, then switching to stage { node { ... } } might be better, as long as the time gained in parallelization is not lost in stashing.

Update:

I tested the exact effect of swapping nesting on one of our builds. with a bunch of stages inside a node, the total build time is just over one minute. With a node inside each stage, total build time is almost five minutes. Big difference.

Build stage timing example

like image 32
Samuel Neff Avatar answered Oct 05 '22 17:10

Samuel Neff