Jenkins pipeline java.io.NotSerializableException java.util.regex.Matcher error even with @NonCPS

Question

I am using @NonCPS in front of my Jenkinsfile function which performs a regex match and i'm still getting java.io.NotSerializableException java.util.regex.Matcher error even with the @NonCPS annotation.

Note, it calls the function many times and the exception only occurs once a match is actually made.

Here is my code:

@NonCPS
def extractEndTime(logLine) {
    def MY_REGEX = /.*(20[0-9]{2}-[0-9]{2}-[0-9]{2}).([0-9]{2}:[0-9]{2}:[0-9]{2}).*\"\w+\"\sthe text\s(\w+)\./
    m = (logLine =~ TEST_LOGLINE_END_REGEX)
    if (m.count) {
        return [m[1],m[2],m[3]]
    } else {
        return null
    }
}

The output when doing a jenkins build:

GitHub has been notified of this commit’s build result
java.io.NotSerializableException: java.util.regex.Matcher
    at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:860)
    at org.jboss.marshalling.river.BlockMarshaller.doWriteObject(BlockMarshaller.java:65)
    at org.jboss.marshalling.river.BlockMarshaller.writeObject(BlockMarshaller.java:56)
    at org.jboss.marshalling.MarshallerObjectOutputStream.writeObjectOverride(MarshallerObjectOutputStream.java:50)
    at org.jboss.marshalling.river.RiverObjectOutputStream.writeObjectOverride(RiverObjectOutputStream.java:179)
    at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:344)
    at java.util.LinkedHashMap.internalWriteEntries(LinkedHashMap.java:333)
    at java.util.HashMap.writeObject(HashMap.java:1354)
    at sun.reflect.GeneratedMethodAccessor116.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at org.jboss.marshalling.reflect.SerializableClass.callWriteObject(SerializableClass.java:271)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:976)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
    at org.jboss.marshalling.river.BlockMarshaller.doWriteObject(BlockMarshaller.java:65)
    at org.jboss.marshalling.river.BlockMarshaller.writeObject(BlockMarshaller.java:56)
    at org.jboss.marshalling.MarshallerObjectOutputStream.writeObjectOverride(MarshallerObjectOutputStream.java:50)
    at org.jboss.marshalling.river.RiverObjectOutputStream.writeObjectOverride(RiverObjectOutputStream.java:179)
    at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:344)
    at com.cloudbees.groovy.cps.SerializableScript.writeObject(SerializableScript.java:26)
    at sun.reflect.GeneratedMethodAccessor145.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at org.jboss.marshalling.reflect.SerializableClass.callWriteObject(SerializableClass.java:271)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:976)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
    at org.jboss.marshalling.river.BlockMarshaller.doWriteObject(BlockMarshaller.java:65)
    at org.jboss.marshalling.river.BlockMarshaller.writeObject(BlockMarshaller.java:56)
    at org.jboss.marshalling.MarshallerObjectOutputStream.writeObjectOverride(MarshallerObjectOutputStream.java:50)
    at org.jboss.marshalling.river.RiverObjectOutputStream.writeObjectOverride(RiverObjectOutputStream.java:179)
    at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:344)
    at java.util.HashMap.internalWriteEntries(HashMap.java:1777)
    at java.util.HashMap.writeObject(HashMap.java:1354)
    at sun.reflect.GeneratedMethodAccessor116.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at org.jboss.marshalling.reflect.SerializableClass.callWriteObject(SerializableClass.java:271)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:976)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
    at org.jboss.marshalling.AbstractObjectOutput.writeObject(AbstractObjectOutput.java:58)
    at org.jboss.marshalling.AbstractMarshaller.writeObject(AbstractMarshaller.java:111)
    at org.jenkinsci.plugins.workflow.support.pickles.serialization.RiverWriter.writeObject(RiverWriter.java:132)
    at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.saveProgram(CpsThreadGroup.java:433)
    at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.saveProgram(CpsThreadGroup.java:412)
    at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:357)
    at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$100(CpsThreadGroup.java:78)
    at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:236)
    at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:224)
    at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:63)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:112)
    at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
Caused by: an exception which occurred:
    in field delegate
    in field closures
    in object org.jenkinsci.plugins.workflow.cps.CpsThreadGroup@42b37962
Finished: FAILURE

Answer 1

Jenkins require all variables to be serializable because the state of the pipeline is periodically saved to disk in case of interrupts like a server restarts. This feature allows pipelines to maintain their state and continue even after the server is restarted. Variables of type Matcher are not serializable and require some additional work by the developer.

From jenkinsci/pipeline-plugin Serializing Local Variables:

However the safest approach is to isolate use of nonserializable state inside a method marked with the annotation @NonCPS. Such a method will be treated as “native” by the Pipeline engine, and its local variables never saved.

The code example provided:

@NonCPS
def version(text) {
  def matcher = text =~ '<version>(.+)</version>'
  matcher ? matcher[0][1] : null
}

Additional material backing this can be found on Pipeline Groovy Plugin Technical Design, here they discuss more technical details and behavior of methods marked with @NonCPS.

Pipeline scripts may mark designated methods with the annotation @NonCPS. These are then compiled normally (except for sandbox security checks), and so behave much like “binary” methods from the Java Platform, Groovy runtime, or Jenkins core or plugin code. @NonCPS methods may safely use non-Serializable objects as local variables, though they should not accept nonserializable parameters or return or store nonserializable values. You may not call regular (CPS-transformed) methods, or Pipeline steps, from a @NonCPS method, so they are best used for performing some calculations before passing a summary back to the main script. Note in particular that @Overrides of methods defined in binary classes, such as Object.toString(), should in general be marked @NonCPS since it will commonly be binary code calling them.

See: Serializing Local Variables and Pipeline Groovy Plugin Technical Design

Answer 2

Probably due to the m variable scope. Try restrict it with def like this:

def m = (logLine =~ TEST_LOGLINE_END_REGEX)

Without def a variable is created in a global script binding and hence still exists after exiting from the method.

Answer 3

The following works (note the [0]):

def m = (it =~ /abcd/)[0]

but this results in a java.io.NotSerializableException:

def m = (it =~ /abcd/)

Answer 4

Ran into this recently. One way to avoid this is to simply set the local matcher-object to null after extracting the parts you need into serializable types. Make sure it's null before doing additional steps/actions.

script {
    def label = 'v1.2.3'  // From git describe etc.
    def regex = '^v([0-9]+).([0-9]+).([0-9]+)$'
    def matcher = label =~ regex

    if (matcher) {
        def tag = matcher[0][0]
        def maj = matcher[0][1] as int
        def min = matcher[0][2] as int
        def patch = matcher[0][3] as int

        matcher = null
        echo "Found ${maj} ${min} ${patch} from ${tag}"
        // Additional steps
    } else {
        matcher = null
        echo "Nothing found"
    }
}

Answer 5

Just for clarity, the following worked for me.

The extra work for the developer, as an example, see the code below.

my.Parameter is NOT Serializable

so I make a method, without a def and annotate it @NonCPS and call that from the node block

  node('MySys') {
    echo 'hello'
    avoidCPS()
    echo 'finished'
  }

  @NonCPS
  avoidCPS () {
    // use my.Parameter here
    my.Parameter p = new my.Parameter()
    ... do some more ...
  }