Jenkins task for remote hosts

Question

In deploy scenario i need to create and run jenkins task on list of hosts, i.e. create something like parametrized task (where ip address is a parameter) or a task on Multijob Plugin with HOST axis, but run by only 2 ones in parallel over multiple hosts.

One of the option could be to run ansible with the list of hosts, but i'd like to see a status per each host separately, and relaunch a jenkins job if needed.

The main option is to use Job DSL Plugin or Pipeline Plugin, but here i need help to understand what classes/methods of dsl groovy code should be used to achieve this.
Can anyone help with it?

Answer 1

Assume that the hosts have been configured as Jenkins slaves already. Assume that hosts are provided in pipeline job parameter HOSTS as whitespace separated list. Following example should get you started:

def hosts_pairs = HOSTS.split().collate(2)

for (pair in host_pairs) {
  def branches = [:]
  for (h in pair) {
    def host = h  // fresh variable per iteration; it will be mutated
    branches[host] = {
      stage(host) {
        node(host) {
          // do the actual job here, e.g. 
          // execute a shell script
          sh "echo hello world"
        }
      }
    }
  }
  parallel branches
}  

Answer 2

A combination of Matrix project and Throttle Concurrent Builds Plugin is possible.

All you need is to setup a single user-defined axis (e.g. "targetHost") with all IP addresses as values and set the desired throttling under "Throttle Concurrent Builds" (please note that you have to enable the "Execute concurrent builds if necessary" option to tell jenkins to allow concurrent execution).

The axis values are available during every child build in the corresponding environment variable (e.g. targetHost).

Below is an example config.xml with simple ping&wait build step:

<?xml version='1.0' encoding='UTF-8'?>
<matrix-project plugin="[email protected]">
  <actions/>
  <description></description>
  <keepDependencies>false</keepDependencies>
  <properties>
    <hudson.plugins.throttleconcurrents.ThrottleJobProperty plugin="[email protected]">
      <maxConcurrentPerNode>2</maxConcurrentPerNode>
      <maxConcurrentTotal>2</maxConcurrentTotal>
      <categories class="java.util.concurrent.CopyOnWriteArrayList"/>
      <throttleEnabled>true</throttleEnabled>
      <throttleOption>project</throttleOption>
      <limitOneJobWithMatchingParams>false</limitOneJobWithMatchingParams>
      <matrixOptions>
        <throttleMatrixBuilds>true</throttleMatrixBuilds>
        <throttleMatrixConfigurations>true</throttleMatrixConfigurations>
      </matrixOptions>
      <paramsToUseForLimit></paramsToUseForLimit>
    </hudson.plugins.throttleconcurrents.ThrottleJobProperty>
  </properties>
  <scm class="hudson.scm.NullSCM"/>
  <canRoam>true</canRoam>
  <disabled>false</disabled>
  <blockBuildWhenDownstreamBuilding>false</blockBuildWhenDownstreamBuilding>
  <blockBuildWhenUpstreamBuilding>false</blockBuildWhenUpstreamBuilding>
  <triggers/>
  <concurrentBuild>true</concurrentBuild>
  <axes>
    <hudson.matrix.TextAxis>
      <name>targetHost</name>
      <values>
        <string>127.0.0.1</string>
        <string>127.0.0.2</string>
        <string>127.0.0.3</string>
        <string>127.0.0.4</string>
        <string>127.0.0.5</string>
      </values>
    </hudson.matrix.TextAxis>
  </axes>
  <builders>
    <hudson.tasks.Shell>
      <command>sleep 7
ping -c 7 $targetHost
sleep 7</command>
    </hudson.tasks.Shell>
  </builders>
  <publishers/>
  <buildWrappers/>
  <executionStrategy class="hudson.matrix.DefaultMatrixExecutionStrategyImpl">
    <runSequentially>false</runSequentially>
  </executionStrategy>
</matrix-project>

Good luck!