Preventing expansion of environment variables in Jenkins job parameters

Question

Problem

I am working on a Jenkins job which accepts some parameters from the user. I am encountering an undesired behaviour: Jenkins appears to be expanding environment variables references within the parameter environment variables before my script has a chance to read them.

If the user enters foo-$BUILD_NUMBER for a parameter, what my script actually sees is something like foo-123; the environment variable is expanded. If input the value contains $$, my script only sees a single $. However, if it contains a $variable which does not exist in the environment, the value is left unmodified (without raising any kind of error).

This is inconvenient because it even happens for password fields, and I typically use a password generator that can include $ characters. I don't want my passwords to potentially be silently mangled.

Example

My initial test case was the following, using the Jenkins Job Builder Groovy DSL.

new BaseJobBuilder(
    jobName: 'example',
    jobBuildName: 'example-${BUILD_NUMBER}',
).build(this).with {
    parameters {
        nonStoredPasswordParam('SERVICE_PASSWORD')
    }
    steps {
        shell('echo "$SERVICE_PASSWORD";')
    }
}

However, for a more reduced test case I created a new Jenkins installation from the jenkins/jenkins:lts Docker image, configured it without any plugins (even the defaults), and created an equivalent job using the web UI.

When I run either of these jobs with the value hello $BUILD_NUMBER $HOME world for my parameter SERVICE_PASSWORD, the output has the variables expanded, instead of the literal value I want.

Started by user jeremy
Building in workspace /var/jenkins_home/workspace/jeremy
[jeremy] $ /bin/sh -xe /tmp/jenkins2451955822062381529.sh
+ echo hello 3 /var/jenkins_home world
hello 3 /var/jenkins_home world
Finished: SUCCESS

Question

Is there any way to access the original parameter values for a Jenkins before they're subject to variable expansion/interpolation, or to otherwise disable or circumvent this behaviour?

How can I accept raw text parameters that may contain $ dollar characters without risk of them being mangled?

Related Links

• JENKINS-10779 Non-existing parameters are not empty but replaced with parameter name
• JENKINS-16143 Jenkins escapes 2 dollar signs in the Parameterized Build fields
• JENKINS-26916 There should be an option to turn on/off variable substitution in build parameters
• Jenkins text parameter - special characters garbled (unwanted variable substitution)

Answer 1

As a workaround, we can add an initial build step that reads all parameters directly and encodes them in base64, then exports the encoded values as new environment variables. Base64-encoded values can't contain any dollar characters $, so they can be safely read by later build steps, which can decode them to get the original value without any expansion.

We implement this with a "System Groovy Script" build step from the Groovy plugin, which runs a custom Groovy script with direct access to the build state (thanks to daspilker for the suggestion). If you're using the DSL, you could add this with a systemGroovyCommand("""…""") call.

import hudson.EnvVars;
import hudson.model.Executor;
import hudson.model.Environment;

def build = Executor.currentExecutor().currentExecutable;

def newVariables = [:];
build.getBuildVariables().each { name, value ->
  def encodedName = name + "_B64";
  def encodedValue = value.bytes.encodeBase64().toString();
  newVariables.put(encodedName, encodedValue);
}

build.getEnvironments().add(Environment.create(new EnvVars(newVariables)))

Your Jenkins administrator may need to approve this script from the In-process Script Approval page the first time it's loaded. Because it already encodes all parameters, you should never need to modify it, and will be able to reuse it in other jobs without reapproval.

Subsequent "Execute Shell" steps will now be able to decode the original value.

set -eu +vx;

echo "directly from environment: $SERVICE_PASSWORD";

SERVICE_PASSWORD="$(echo "$SERVICE_PASSWORD_B64" | base64 --decode)";
echo "via base-64 encoded value: $SERVICE_PASSWORD";

We can use our original test value of hello $BUILD_NUMBER $HOME world to confirm that it works:

directly from environment: hello 12 /var/jenkins_home world
via base-64 encoded value: hello $BUILD_NUMBER $HOME world

---

It is not possible to disable variable expansion when you're reading parameters from environment variables. This behaviour doesn't have to do with parameters specifically, but with how Jenkins handles environment variables in general. After the hudson.model.AbstractBuild.getEnvironment(…) method aggregates all of the environment variable for a build, it applies the hudson.EnvVars.resolve(…) function to perform mutual variable expansion on the contents of all environment variables. Depending on the exact build configuration, it may also use hudson.EnvVars.overrideExpandingAll(…), which takes the additional step of topologically sorting the variables to ensure that non-cyclic references are all expanded in the correct order. The actual string manipulation is performed by hudson.util.replaceMacro(…), which has a comment explaining the unusual handling (non-replacement) of nonexistent variables:

Unlike shell, undefined variables are left as-is (this behavior is the same as Ant.)

These expansions are performed unconditionally, so there's no way to disable it without modifying or replacing several classes in Jenkins.