Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Parallel matrix and global variables without race condition?

I have the following declarative pipeline where I write a global build variable during a parallel matrix, the write in stage Build Detection is probably (wasn't clear to me) a race condition but I am not sure. I have 3 questions regarding the below simple pipeline:

  1. Is it correct that since Build-Detection uses the same agent (note only Build uses a different agent), it is definitely a race condition ?
  2. If I would have one agent for each parallel line, it would not be a race condition as the global build is different in each agent?
  3. Is there a way to make a variable copy of build inside the stage such that it's not global any more?
  4. How should we deal with global variable communicating stuff (for when steps etc) and parallel matrix feature?

Map<String,Boolean> build

pipeline {
  stages {
    stage('Test') {
      failFast false
      matrix {
        axes {
          axis {
            name 'CONTAINER'
            values 'A', 'B'
          }
        }
        stages {
          stage('Build Detection') {
            steps {
              script {
                build[CONTAINER] = CONATAINER == 'A'
                echo "Should Build: ${build[CONTAINER]}"
              }
            }
          }
          stage('Build') {
            agent {
              kubernetes {
                yamlFile '.jenkins/pods/build-kaniko.yaml'
              }
            }
            when {
              beforeAgent true
              expression { return build[CONTAINER] }
            }
            steps {
                echo "BUILDING....."
            }
          }
        }
      }
    }
  }
}

like image 881
Gabriel Avatar asked Sep 11 '25 08:09

Gabriel


1 Answers

  1. No, it has nothing to do with build agents. The JVM that's executing the compiled groovy code is running on the Jenkins master, not a build agent. Therefore, using a global variable is shared by each thread running in the Jenkins master JVM. Whether there's a possible race condition is not related to stages using the same or different build agents.

  2. Same answer as 1.

  3. Yes, simply define a variable using "def" or a specific type in the stage's script block. Just be sure to not reference a new variable without a type because in Groovy that causes it to be declared globally.

  4. Using a map with a key that is specific to each thread like you're doing seems like a good way to me. If you really want to make sure there is no possibility of two unsafe thread operations modifying the map at the same time, then make sure that a threadsafe map is used. You could print out the class of the map to find out what implementation is getting instantiated. I would hope it's something threadsafe like ConcurrentHashMap.

like image 135
UltimateGeek Avatar answered Sep 15 '25 17:09

UltimateGeek