Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Declarative Pipeline with dynamic matrix axis values

Hi I am trying to get a Jenknis-Declarative-Pipeline-Job work.

The Use-Case should be pretty simple:
I want to build multiple Plugins with the same Jenkins-pipeline. To do so I wrote a "JenkinsLibrary" with an interface the Plugins can use for parameters. One of this parameters is the axis-values.

The Problem I have is pretty similar to this reddist post.

I want to set the "values" of the "axis" of the matrix-build from a variable. I am out of Ideas, is this even possible?

So here is my example:
a "JenkinsLibrary" with a file my_library.groovy

def call(Map i_options)
{
// later I will parse the options to set the values, but currently the step before already crashes ...

    def axis_1_values = "axis_1_value_A"
// already tried the following alternatives ...
//    def axis_1_values = 'axis_1_value_A'
//    def axis_1_values = '''axis_1_value_A'''
//    def axis_1_values = ["axis_1_value_A", "axis_2_value_A"]
    pipeline
    {
        agent any
        stages { stage("stage A") { matrix {
            axes {
                axis {
                    name "axis_1"
                    // values "axis_1_value_A" // <- of course this works ...
                    // but I want to read it from a variable
                    values "${axis_1_values}" 
                    // I already tried more variants
                    // values ${axis_1_values}
                    // values axis_1_values
                }
                axis {
                    name "axis_2"
                    values "axis_2_value_A", "axis_2_value_B"
                }
            }
            stages {
                stage("another stage") { steps {
                    echo "hello world from ${axis_1} && ${axis_2}"
                } }
            }
        } } }
    }
}

A Plugin with a file "Jenkinsfile":

@Library("JenkinsLibrary") _

def options = [
    axis_values: "a_axis_value"
]

my_library(options)

I am getting the following error:

[...] Expected string literal but got "${axis_1_values}"

complete log:

    19: Expected string literal but got "${axis_1_values}" @ line 19, column 28.
                       values "${axis_1_values}" 
                              ^

1 error

    at org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:310)
    at org.codehaus.groovy.control.CompilationUnit.applyToPrimaryClassNodes(CompilationUnit.java:1085)
    at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:603)
    at org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:581)
    at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:558)
    at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:298)
    at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:268)
    at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:254)
    at groovy.lang.GroovyClassLoader.recompile(GroovyClassLoader.java:761)
    at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:718)
    at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:787)
    at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:775)
    at org.jenkinsci.plugins.workflow.cps.global.UserDefinedGlobalVariable.getValue(UserDefinedGlobalVariable.java:57)
    at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:113)
    at sun.reflect.GeneratedMethodAccessor729.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
    at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1213)
    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022)
    at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:42)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
    at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:160)
    at org.kohsuke.groovy.sandbox.GroovyInterceptor.onMethodCall(GroovyInterceptor.java:23)
    at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:157)
    at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:142)
    at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:158)
    at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:162)
    at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:17)
    at WorkflowScript.run(WorkflowScript:21)
    at ___cps.transform___(Native Method)
    at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:86)
    at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:113)
    at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:83)
    at sun.reflect.GeneratedMethodAccessor500.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
    at com.cloudbees.groovy.cps.impl.LocalVariableBlock$LocalVariable.get(LocalVariableBlock.java:39)
    at com.cloudbees.groovy.cps.LValueBlock$GetAdapter.receive(LValueBlock.java:30)
    at com.cloudbees.groovy.cps.impl.LocalVariableBlock.evalLValue(LocalVariableBlock.java:28)
    at com.cloudbees.groovy.cps.LValueBlock$BlockImpl.eval(LValueBlock.java:55)
    at com.cloudbees.groovy.cps.LValueBlock.eval(LValueBlock.java:16)
    at com.cloudbees.groovy.cps.Next.step(Next.java:83)
    at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:174)
    at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:163)
    at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:129)
    at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:268)
    at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:163)
    at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$001(SandboxContinuable.java:18)
    at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:51)
    at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:185)
    at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:405)
    at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$400(CpsThreadGroup.java:96)
    at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:317)
    at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:281)
    at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:67)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131)
    at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
    at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
Finished: FAILURE

My System: Jenkins-Version: Jenkins ver. 2.190.1
Declarative-Plugin-Version 1.5.0
Declarative Agent API: 1.1.1
Declarative Extension Points API: 1.5.0
(many other plugins, I think this are the important ones)

UPDATE
I still do not have a solution. But when I define a variable like this, I understand that I try to insert a "GStringImpl" to that. But I am unable to cast it as java-string.

def _str_1 = "a string"
def _str_2 = "${_str_1}"
println _str_1.class             // -> class java.lang.String
println _str_2.class             // -> class org.codehaus.groovy.runtime.GStringImpl
def _str_3 = "${_str_1}" as java.lang.String  
println _str_3.class             // -> class java.lang.String

But I still don't know how I can add a single value, because if I do

values "${axis_1_values}" as as java.lang.String     // error-msg: Expected string literal but got ${as}  
values ("${axis_1_values}" as java.lang.String)      // error-msg: Expected string literal but got ${"${axis_1_values}"  
values axis_1_values.toString()                      // error-msg: Method calls on objects not allowed outside "script" blocks.
values "${axis_1_values}".toString()                 // error-msg: Expected a symbol @ line 26, column 28.

This casting stuff may work for a single value, but I want to be able to add a list of values, like its possible in the default example which works ...
values "axis_2_value_A", "axis_2_value_B"

I am wondering, that this works in the "docker-agent"-part of the declarative pipeline.

agent { 
    docker {
        image "${_image}:latest"
        label "${_label}"
    }
}

But I in the code, I realized that the parsing of the docker-label is implemented separately ...

I have still no Ideas how to reach that goal within the matrix / axis features of the declarative pipeline.

(funfact: one of the main reasons why even doing this that the visualisation in the blue-ocean view for scripted-pipelines is still buggy and the issue seems to be ignored ...
https://issues.jenkins-ci.org/browse/JENKINS-53751 )

like image 203
tharilya Avatar asked Jan 21 '20 15:01

tharilya


People also ask

Is Declarative pipeline better than scripted?

There are no differences in the runtime performance, scalability and problem solvability perspective in the declarative vs. scripted pipeline debate. They only differ in the syntactic approach used to achieve an end goal.

What directive do you use for a Declarative pipeline?

The agent directive specifies where the entire Pipeline, or a specific stage, will execute in the Jenkins environment depending on where the agent directive is placed. The directive must be defined at the top-level inside the pipeline block, but stage-level usage is optional.

What is a pipeline Matrix?

The Declarative Pipeline Matrix section allows users to specify a list stages once and then run that same list in parallel on multiple configurations. The new matrix section executes a set of one or more Pipeline stages multiple times - once for every combination defined in the matrix.


1 Answers

Dynamic axis values are not supported in Declarative Pipeline as of v1.6.0 of the plugin. The error message says it expects a "literal string" value. Perhaps the message should also say, "Template strings, arrays, variables, or function calls are not allowed."

This is by design, intended to keep pipeline authors from getting bitten by some internal assumptions of the pipeline engine. It might be possible to change but will require careful planning and testing.

Please file a JIRA at https://issues.jenkins.io/ and/or join the Jenkins Pipeline Authoring SIG and help implement this feature.

like image 160
BitwiseMan Avatar answered Oct 14 '22 22:10

BitwiseMan