NullPointerException in native java code while performing parallelStream.forEach(..)

Question

I have the following exception (the stacktrace):

java.lang.NullPointerException
at sun.reflect.GeneratedConstructorAccessor171.newInstance(Unknown Source) ~[?:?]
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_40]
at java.lang.reflect.Constructor.newInstance(Constructor.java:422) ~[?:1.8.0_40]
at java.util.concurrent.ForkJoinTask.getThrowableException(ForkJoinTask.java:598) ~[?:1.8.0_40]
at java.util.concurrent.ForkJoinTask.reportException(ForkJoinTask.java:677) ~[?:1.8.0_40]
at java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:735) ~[?:1.8.0_40]
at java.util.stream.ForEachOps$ForEachOp.evaluateParallel(ForEachOps.java:160) ~[?:1.8.0_40]
at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateParallel(ForEachOps.java:174) ~[?:1.8.0_40]
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:233) ~[?:1.8.0_40]
at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418) ~[?:1.8.0_40]
at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:583) ~[?:1.8.0_40]
at com.tradair.tnet.services.trades.TradeService.updateUnrealizedPNL(TradeService.java:173) ~[tnet.jar:5.1.1.0-SNAPSHOT]

which starts at my TradeService class here:

    public void updateUnrealizedPNL(Set<Org> orgsToCaluclate, Set<Org> orgsToSendUpdate) {
    orgsToCaluclate.parallelStream().forEach(o -> {
        pnlService.updateMidPrices(o);
        Collection<SystemTradeOrder> allLiveTradesByOrgId = tradesRepository.getAllLiveTradesByOrgId(o.getId());
        updateUnrealizedPNL(o, allLiveTradesByOrgId);
    });

    // more code ....

So it looks like the exception is thrown inside java native code while running forEach(..) method. I mean, the NullPointerException isn't thrown from my own code - not from my consumer function which appears as the argument in forEach(..) method.

I double checked that there is no modification to the orgsToCaluclate set when this piece of code runs.

This is the initialization of orgsToCaluclate:

        Set<Org> orgsToCaluclate = getMarginOrgs();
        orgsToCaluclate = orgsToCaluclate.stream()
                .filter(org -> !isOrgInCloseout(org.getId())).collect(Collectors.toSet());

Any thoughts?..

Answer 1

We got used to say that an exception’s stack trace reflects “where it happened”, but that’s an imprecise statement. An exception’s stack trace usually reflects where its instance has been created.

When we have code of the form,

1   String s=null;
2   s.length();

The JRE will create an instance of NullPointerException when we try to dereference null for invoking the method length(), so its stack trace will report line 2.

However, when we have the following code

1   String s=null;
2   if(s == null) {
3       RuntimeException rt=new NullPointerException();
4       throw rt;
5   }

The stack trace will not report where the erroneous condition has been detected (line 2) nor where the exception has been thrown (line 4) but where the instance has been created, in line 3.

For most practical cases, these places are close enough to make no significant difference, but here, we have an extraordinary situation.

As tonakai has pointed out, the ForkJoinTask will create a new instance of an already encountered exception via Reflection, as we can see in its source code when the threads mismatch.

When it succeeds, its stack trace will precisely reflect where the new exception instance has been created, which is in some generated code performing the reflective instance creation. Of course, this successful creation can’t be distinguished from the situation when the JRE creates an exception due to an error condition when executing the same code.

But when we look closer at the source code, we see that the entire reflective creation is enclosed with a

584             try {
…
604             } catch (Exception ignore) {
605             }

block. So if the operation really failed, no exception was visible. Instead, the code had fallen over to return the original exception. This indicates that the reflective code didn’t fail, but instead we see the successfully, reflectively created NullPointerException instance returned by getThrowableException() and later on deliberately throw by ForkJoinTask to report that there was a NullPointerException in another thread during the processing.

But this code initializes the cause of the new exception to point to the original one. E.g. the following code:

import java.util.stream.IntStream;

public class Main
{
    public static void main(String[] args) {
        Thread main=Thread.currentThread();
        IntStream.range(0, 1000).parallel().forEach(i -> {
            if(Thread.currentThread()!=main)
                throw new NullPointerException();
        });
    }
}

prints

Exception in thread "main" java.lang.NullPointerException
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
    at java.util.concurrent.ForkJoinTask.getThrowableException(ForkJoinTask.java:598)
    at java.util.concurrent.ForkJoinTask.reportException(ForkJoinTask.java:677)
    at java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:735)
    at java.util.stream.ForEachOps$ForEachOp.evaluateParallel(ForEachOps.java:160)
    at java.util.stream.ForEachOps$ForEachOp$OfInt.evaluateParallel(ForEachOps.java:189)
    at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:233)
    at java.util.stream.IntPipeline.forEach(IntPipeline.java:404)
    at java.util.stream.IntPipeline$Head.forEach(IntPipeline.java:560)
    at Main.main(Main.java:7)
Caused by: java.lang.NullPointerException
    at Main.lambda$main$0(Main.java:9)
    at java.util.stream.ForEachOps$ForEachOp$OfInt.accept(ForEachOps.java:205)
    at java.util.stream.Streams$RangeIntSpliterator.forEachRemaining(Streams.java:110)
    at java.util.Spliterator$OfInt.forEachRemaining(Spliterator.java:693)
    at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
    at java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:291)
    at java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:731)
    at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
    at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
    at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
    at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)

so you are still able to recognize what happened. You just have to pay attention to the cause. Since the stack trace in your question doesn’t look like the typical Throwable.printStackTrace() output, it might be the code which produced this output which ignored the cause property of the exception.

---

As an addendum, we can check what happens if that recreation really fails using a custom exception type:

import java.util.stream.IntStream;

public class Main
{
    public static class CustomException extends RuntimeException {
        public CustomException() {
            System.err.println("will deliberately fail");
            throw new NullPointerException();
        }
        private CustomException(String message) {
            super(message);
        }
    }
    public static void main(String[] args) {
        Thread main=Thread.currentThread();
        IntStream.range(0, 1000).parallel().forEach(i -> {
            if(Thread.currentThread()!=main)
                throw new CustomException("forced failure");
        });
    }
}

will print

will deliberately fail
Exception in thread "main" Main$CustomException: forced failure
    at Main.lambda$main$0(Main.java:18)
    at java.util.stream.ForEachOps$ForEachOp$OfInt.accept(ForEachOps.java:205)
    at java.util.stream.Streams$RangeIntSpliterator.forEachRemaining(Streams.java:110)
    at java.util.Spliterator$OfInt.forEachRemaining(Spliterator.java:693)
    at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
    at java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:291)
    at java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:731)
    at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
    at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
    at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
    at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)

showing that the NullPointerException thrown during the reflective recreation via the default constructor stays unreported and the original exception from the other thread is thrown directly instead.

Answer 2

looking at the stacktrace it looks like there is an exception happening in your code, and it is trying to get a new instance of that exception but fails to do so because of NullPointerException, can you check your code if any of the code that is called in that foreach throws any exception and make sure all constructors of those exceptions are correct

    at java.lang.reflect.Constructor.newInstance(Constructor.java:422) ~[?:1.8.0_40]
at java.util.concurrent.ForkJoinTask.getThrowableException(ForkJoinTask.java:598) ~[?:1.8.0_40]
at java.util.concurrent.ForkJoinTask.reportException(ForkJoinTask.java:677) ~[?:1.8.0_40]
at java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:735) ~[?:1.8.0_40]