Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Problem with XmlDecoder and Akka Stream only inside Docker container

Tags:

I've faced a problem using XmlDecoder in AkkaStream only in an app running in a Docker container.


Error description

java.lang.ClassNotFoundException: com/example/xmldecoder/FileDto
Continuing ...
java.lang.ClassNotFoundException: com/example/xmldecoder/FileDto
Continuing ...
java.lang.NoSuchMethodException: <unbound>=XMLDecoder.new();
Continuing ...
java.lang.NoSuchMethodException: <unbound>=XMLDecoder.new();
Continuing ...
java.lang.IllegalStateException: The outer element does not return value
Continuing ...
java.lang.IllegalStateException: The outer element does not return value
Continuing ...
java.lang.IllegalStateException: The outer element does not return value
Continuing ...
java.lang.IllegalStateException: The outer element does not return value
Continuing ...
2019-05-22 09:42:29.145 ERROR 1 --- [onPool-worker-5] com.example.xmldecoder.FileReader        : Unexpected exception in load file, {} 
java.lang.ArrayIndexOutOfBoundsException: Index 0 out of bounds for length 0
    at java.desktop/java.beans.XMLDecoder.readObject(XMLDecoder.java:251) ~[na:na]
    at com.example.xmldecoder.FileReader.lambda$loadFile$0(XmlDecoderApplication.java:66) ~[classes!/:0.0.1-SNAPSHOT]
    at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1700) ~[na:na]
    at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.exec(CompletableFuture.java:1692) ~[na:na]
    at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290) ~[na:na]
    at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020) ~[na:na]
    at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656) ~[na:na]
    at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594) ~[na:na]
    at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:177) ~[na:na]

There is a couple of conditions to be met:

  1. The error occurs only within Docker, when you run the code on a non-containerized host everything is ok
  2. Problem is only when you use XmlDecoder, reading the file line by line using BufferedReader works fine
  3. When you limit docker CPUs (--cpus=1) the error doesn't occur
  4. When you use an ExecutorService instead of Akka Streams the error doesn't occur
  5. I've tried to use some docker flags that help with JDK problems (UseContainerSupport, ActiveProcessorCount) but it didn't help

Code

Runnable example available here

Problematic code below:

@Slf4j
@RequiredArgsConstructor
class FileReader {

private final ActorSystem system;
private final ReadJob readJob;

public NotUsed loadFiles() {
    List<String> paths = listFiles(readJob);
    return Source.from(paths)
            .via(Flow.of(String.class).mapAsync(5, p -> loadFile(p)))
            .to(Sink.foreach(System.out::println)).run(ActorMaterializer.create(system));
}

private CompletionStage<String> loadFile(String filePath) {
    return CompletableFuture.supplyAsync(() -> {
        try {
            FileInputStream fis2 = new FileInputStream(filePath);
            BufferedInputStream bis2 = new BufferedInputStream(fis2);
            XMLDecoder xmlDecoder = new XMLDecoder(bis2);
            FileDto mb = (FileDto) xmlDecoder.readObject();
            log.info("Decoder: {}", mb);
            return mb.toString();
        } catch (Exception e) {
            log.error("Unexpected exception in load file, {}", e);
            throw new RuntimeException("Unexpected exception in load file", e);
        }
    });
}

private List<String> listFiles(ReadJob readJob) {
    File folder = new File(readJob.getHolderDirPath().toString());
    File[] listOfFiles = folder.listFiles();
    log.info(listOfFiles.toString());
    return Stream.of(listOfFiles).map(File::getAbsolutePath).collect(Collectors.toList());
}

}

Can be run, e.g., this way:

@SpringBootApplication
@EnableScheduling
@Slf4j
public class XmlDecoderApplication {

private Path holderPath = Paths.get("opt", "files_to_load");

public static void main(String[] args) {
    SpringApplication.run(XmlDecoderApplication.class, args);
}

@Scheduled(fixedDelay = 30000, initialDelay = 1000)
public void readFiles() {
    FileReader reader = new FileReader(ActorSystem.create(), new ReadJob(holderPath));
    reader.loadFiles();
}
}

I suppose the root cause is somewhere between host <-> docker <-> java

Thanks in advance for any help with this

like image 686
rkarczmarczyk Avatar asked May 22 '19 10:05

rkarczmarczyk


1 Answers

The example code worked for me with the following modification: replace the line

XMLDecoder xmlDecoder = new XMLDecoder(bis2);

with

XMLDecoder xmlDecoder = new XMLDecoder(bis2, null, null, FileDto.class.getClassLoader());

i.e. effectively force the XMLDecoder to use the precise classloader used to load the class in question. But as for why it only occurs

  1. in Docker
  2. if --cpus is set to sth greater than 1
  3. with Akka Streams
  4. with specific JDK versions

– I only have some (mostly) uneducated guesses.

like image 72
Michal M Avatar answered Oct 14 '22 07:10

Michal M