Running Jenkins job simultaneously on all nodes

Question

TLDR: I want to be able to run job simultaneously on multiple nodes in Jenkins pipeline. [ for example - build application x on nodes dev, test & staging nodes based on aws ]

I have a large group of nodes with the same label. I would like to be able to run a job in Jenkins that executes on all of the nodes with the same label as well as doing so simultaneously.

I saw a suggestion to use the matrix configuration option in Jenkins, but I can only think of one axis (the label group). When I try and run the job, it seems like it only executes once instead of 300 times (1 for each of the nodes in that label group).

What should my other axis be? Or...is there some plugin to do this? I had tried the NodeLabel Parameter Plugin, and choosing "run on all available online nodes", but it does not seem to run the jobs simultaneously.

Answer 1

1. Install
   • Parameterized Trigger Plugin
   • NodeLabel Parameter Plugin
2. For the job you want to run, enable Execute concurrent builds if necessary
3. Create another job besides the job you want to run on all slaves and configure it
   • Build > Add build step > Trigger/call builds on other projects
     • Add ParameterFactories > All Nodes for Label Factory > Label: the label of the nodes

Answer 2

The matrix build will work; use "Slaves" as the axis and expand the "Individual nodes" list to select all of your nodes.

Note that you will need to update the selection every time you add or remove a slave.

For a more maintainable solution, you could use the Job DSL plugin to set up a seed job that has the template for the build, then loops over each slave and creates a new job with the build label set to the name of the slave.

Answer 3

There is two plugins that you need: Paramitrized Trigger Plugin to be able to trigger other jobs as build step of your main job, and NodeLabel Plugin (read the BuildParameterFactory section for descrition of what you need) to specify the label.

Answer 4

The best and easiest way to accomplish this is using Elastic Axis plugin.
1. Install the pulgin.
2. Create a Multi Configuration job.(Install if not present)
3. In the job configuration you can find new axis added as Elastic axis. Add the label as shown below to get the job run on multiple slaves.