Reusing stages of a jenkins pipeline in multiple jobs

Question

My team is moving to Jenkins 2 and I am using the pipeline plugin so that our build can live in our repository. Because getting repositories allocated has lots of overhead in our company we have a single respository with many sub-projects & sub-modules in it.

What I want is separate builds and reporting of Junit/checkstyle/etc reports for each sub-module as well as a final "build and deploy" step for each sub-project putting it all together.

My current plan is to create separate jobs for each sub-module so that they get their own junit/checkstyle/etc reports page. Then have a multi-job project to orchestrate the sub-module builds for the sub-projects. Since all of the sub-projects are simple jar builds, I want to put bulk of the logic in a common file, lets call it JenkinsfileForJars at the root of the sub-project. So the repo structure is

• sub-project
  • JenkinsfileForJars.groovy
  • sub-moduleA
    • Jenkinsfile
  • sub-moduleB
    • Jenkinsfile

My Jenkinsfile contains

def submoduleName = "submoduleA"
def pipeline
node {

    pipeline = load("${env.WORKSPACE}/subproject/JenkinsfileForJars.groovy")

}
pipeline.build()
pipeline.results()

And my JenkinsfileForJars contains

def build() {

    stage('Build') {
        // Run the maven build
        dir("subproject") {
            sh "./gradlew ${submoduleName}:build"
        }

    }
}
def results() {

    stage('Results') {
        dir("subproject/${submoduleName}") {
            junit 'build/test-results/TEST-*.xml'
            archive 'build/libs/*.jar'
            publishHTML([allowMissing: false, alwaysLinkToLastBuild: false, keepAll: false, reportDir: 'build/reports/cobertura/', reportFiles: 'frame-summary.html', reportName: 'Cobertura Report'])
            publishHTML([allowMissing: false, alwaysLinkToLastBuild: false, keepAll: false, reportDir: 'build/reports/findbugs/', reportFiles: 'main.html', reportName: 'Fidbugs Report'])
            publishHTML([allowMissing: false, alwaysLinkToLastBuild: false, keepAll: false, reportDir: 'build/reports/pmd/', reportFiles: 'main.html', reportName: 'PMD Report'])
            step([$class: 'CheckStylePublisher', pattern: 'build/reports/checkstyle/main.xml', unstableTotalAll: '200', usePreviousBuildAsReference: true])
        }
    }

}

return this;

When I run the Jenkinsfile above I get the following error:

Running on master in /var/lib/jenkins/workspace/jobA
[Pipeline] {
[Pipeline] load
[Pipeline] { (/var/lib/jenkins/workspace/jobA/subproject/JenkinsfileForJars.groovy)
[Pipeline] }
[Pipeline] // load
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
java.lang.NullPointerException: Cannot invoke method build() on null object

As far as I can tell, I am following what is shown in the documents for loading manual scripts and the example given for a loaded script. I do not understand why my script is null after the load command.

How do I get my Jenkinsfile to load JenkinsfileForJars.groovy?

Answer 1

The problem is related to the SCM checkout as mentioned by Blake Mitchell in the comment above. Since you are loading your groovy functions from a submodule, you will need to checkout the submodule first, preferably on a build agent /slave, if you would like to keep only bare repos on the master.

def pipeline
node( 'myAgentLabel' ) {
    stage ( 'checkout SCM' ) {
        checkout([
            $class: 'GitSCM'
            ,branches: scm.branches
            ,extensions: scm.extensions 
                + [[ $class: 'SubmoduleOption', disableSubmodules: false, parentCredentials: true, recursiveSubmodules: true, reference: '', trackingSubmodules: false]]
            ,doGenerateSubmoduleConfigurations: false
            ,userRemoteConfigs: scm.userRemoteConfigs
        ])
        pipeline = load( "${env.WORKSPACE}/path/to/submodule/myGroovyFunctions.grooovy" )
    }
    pipeline.build()
}

Note that in the checkout example, access to scm.* attributes also needs to be whitelisted by an administrator in Jenkins (In-process script approval)