deadlock when run two pipelines parallel in jenkins

Question

Currently I have the following problem. I have written a Jenkinsfile to Build my Repository in a Pipeline. Each Repository has their own Pipeline in a Multibranch Pipeline. Whenever I push the Repository the Pipeline start to Work.

For the building I have one Agent with two nodes. When the Multibranch Pipeline is running, the Multibranch Pipeline uses one node the execute a single Pipeline and the second node is used by the currently executing pipeline to run a single Job.

When two Pipelines run to the same time both Pipelines use one node. But now the Problem is both Pipelines can't start any Jobs since all nodes are occupied. At this time I have a deadlock since both pipelines are waiting for a free node for their jobs.

I have tried to set "disableConcurrentBuilds()", but this only blocks the Pipeline with the same name. Pipelines with different names in the Multibranch Pipeline can run concurrently.

A second try is to set Build Blocker Plugin with this code in the Jenkinsfile.

properties([
    [$class: 'BuildBlockerProperty',
     blockLevel: 'GLOBAL',
     blockingJobs: '*pipeline_Test*',
     scanQueueFor: 'ALL',
     useBuildBlocker: true],
   disableConcurrentBuilds()
   ])

But then I get this Error message.

WorkflowScript: 30: Invalid option type "properties". Valid option types: [buildDiscarder, catchError, checkoutToSubdirectory, disableConcurrentBuilds, disableResume, durabilityHint, lock, newContainerPerStage, overrideIndexTriggers, retry, script, skipDefaultCheckout, skipStagesAfterUnstable, timeout, waitUntil, withContext, withCredentials, withEnv, ws] @ line 30, column 4

How can I set the BuildBlockerProperty in the Jenkinsfile for the entire pipeline? Is there a other way to block all other pipelines so long the pipeline is running?

Thank you for help.

Answer 1

Struggled with the same issue.

You didn't give your Jenkinsfile but if your pipeline requires 1 executor node to run the pipeline and 1 additional to execute the job it is probably that you set an agent at pipeline level and also at stage level like

pipeline {
  agent any

  stage('Build') {
    agent {
      label 'my label'
    }

    steps {
      ...
    }
  }

  post {
    always {
      ...
    }
  }
}

as I did. Of course your specific agent setting can be very different, but the agent is set at 2 levels. The reason I had to specify a pipeline level agent in addition to the stage was because otherwise my post step wouldn't run.

If your pipeline requires specific agent settings (labels, docker images, ...) at stage level, it is best practice to set agent none at pipeline level:

pipeline {
  agent none

  stage('Build') {
    agent {
      label 'my label'
    }

    steps {
      ...
    }
  }
}

This way only 1 executor node is needed thus preventing the deadlock.

If like me you need a post step you can run it like

pipeline {
  agent none

  stage('Build') {
    agent {
      label 'my other label'
    }

    steps {
      ...
    }
  }

  post {
    always {
      node('label') {
        ...
      }
    }
  }
}

to provide it with a node. The label is mandatory, haven't found a way to run it on any node. Adding the node{} bloc allows the post block to run with agent none set at pipeline level.

This worked for me and might be a solution for the OP. Not enough information was given in the OP to know the specific configuration of the pipeline.