Logo Questions Linux Laravel Mysql Ubuntu Git Menu
HTML CSS JAVASCRIPT SQL PYTHON PHP BOOTSTRAP JAVA JQUERY R React Kotlin
 

Apache Flink: ClassNotFoundException on remote cluster

Tags:

apache-flink

I'm running into a ClassNotFoundException only when I run my application on a standalone cluster (using flink cli). If i directly run the main class in my IDE, it's working fine. But using the following cli, it throws error:

./bin/flink run /mnt/MykafkaConsumerApp.jar

This is a spring boot app that I've packaged using maven (Apache Maven 3.3.9). I've verified the class is present in the jar as well.

The execute() call happens inside MyWikiAnalysis spring bean on PostConstruct.

Following are the cluster logs:

2019-05-02 14:54:32,984 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - --------------------------------------------------------------------------------
2019-05-02 14:54:32,986 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  Starting StandaloneSessionClusterEntrypoint (Version: 1.8.0, Rev:4caec0d, Date:03.04.2019 @ 13:25:54 PDT)
2019-05-02 14:54:32,986 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  OS current user: abhishek
2019-05-02 14:54:32,986 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  Current Hadoop/Kerberos user: <no hadoop dependency found>
2019-05-02 14:54:32,987 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  JVM: Java HotSpot(TM) 64-Bit Server VM - Oracle Corporation - 1.8/25.112-b15
2019-05-02 14:54:32,987 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  Maximum heap size: 981 MiBytes
2019-05-02 14:54:32,987 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  JAVA_HOME: /usr/local/java/jdk1.8.0_112
2019-05-02 14:54:32,987 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  No Hadoop Dependency available
2019-05-02 14:54:32,987 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  JVM Options:
2019-05-02 14:54:32,987 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -     -Xms1024m
2019-05-02 14:54:32,987 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -     -Xmx1024m
2019-05-02 14:54:32,987 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -     -Dprofile=dev
2019-05-02 14:54:32,988 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -     -Dlog.file=/mnt/installation/flink-1.8.0/log/flink-abhishek-standalonesession-4-abhishek-hp-probook-430-g3.log
2019-05-02 14:54:32,988 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -     -Dlog4j.configuration=file:/mnt/installation/flink-1.8.0/conf/log4j.properties
2019-05-02 14:54:32,988 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -     -Dlogback.configurationFile=file:/mnt/installation/flink-1.8.0/conf/logback.xml
2019-05-02 14:54:32,988 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  Program Arguments:
2019-05-02 14:54:32,988 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -     --configDir
2019-05-02 14:54:32,988 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -     /mnt/installation/flink-1.8.0/conf
2019-05-02 14:54:32,988 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -     --executionMode
2019-05-02 14:54:32,988 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -     cluster
2019-05-02 14:54:32,988 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  Classpath: /mnt/installation/flink-1.8.0/lib/MykafkaConsumerApp.jar:/mnt/installation/flink-1.8.0/lib/log4j-1.2.17.jar:/mnt/installation/flink-1.8.0/lib/slf4j-log4j12-1.7.15.jar:/mnt/installation/flink-1.8.0/lib/flink-dist_2.11-1.8.0.jar:::
2019-05-02 14:54:32,988 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - --------------------------------------------------------------------------------
2019-05-02 14:54:32,990 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - Registered UNIX signal handlers for [TERM, HUP, INT]
2019-05-02 14:54:33,013 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: env.java.opts, "-Dprofile=dev"
2019-05-02 14:54:33,014 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: jobmanager.rpc.address, localhost
2019-05-02 14:54:33,014 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: jobmanager.rpc.port, 6123
2019-05-02 14:54:33,014 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: jobmanager.heap.size, 1024m
2019-05-02 14:54:33,014 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: taskmanager.heap.size, 1024m
2019-05-02 14:54:33,015 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: taskmanager.numberOfTaskSlots, 1
2019-05-02 14:54:33,015 INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration property: parallelism.default, 1
2019-05-02 14:54:33,103 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - Starting StandaloneSessionClusterEntrypoint.
2019-05-02 14:54:33,103 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - Install default filesystem.
2019-05-02 14:54:33,110 INFO  org.apache.flink.core.fs.FileSystem                           - Hadoop is not in the classpath/dependencies. The extended set of supported File Systems via Hadoop is not available.
2019-05-02 14:54:33,125 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - Install security context.
2019-05-02 14:54:33,134 INFO  org.apache.flink.runtime.security.modules.HadoopModuleFactory  - Cannot create Hadoop Security Module because Hadoop cannot be found in the Classpath.
2019-05-02 14:54:33,146 INFO  org.apache.flink.runtime.security.SecurityUtils               - Cannot install HadoopSecurityContext because Hadoop cannot be found in the Classpath.
2019-05-02 14:54:33,147 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - Initializing cluster services.
2019-05-02 14:54:33,368 INFO  org.apache.flink.runtime.rpc.akka.AkkaRpcServiceUtils         - Trying to start actor system at localhost:6123
2019-05-02 14:54:33,807 INFO  akka.event.slf4j.Slf4jLogger                                  - Slf4jLogger started
2019-05-02 14:54:33,857 INFO  akka.remote.Remoting                                          - Starting remoting
2019-05-02 14:54:33,967 INFO  akka.remote.Remoting                                          - Remoting started; listening on addresses :[akka.tcp://flink@localhost:6123]
2019-05-02 14:54:33,978 INFO  org.apache.flink.runtime.rpc.akka.AkkaRpcServiceUtils         - Actor system started at akka.tcp://flink@localhost:6123
2019-05-02 14:54:33,999 INFO  org.apache.flink.configuration.Configuration                  - Config uses fallback configuration key 'jobmanager.rpc.address' instead of key 'rest.address'
2019-05-02 14:54:34,011 INFO  org.apache.flink.runtime.blob.BlobServer                      - Created BLOB server storage directory /tmp/blobStore-7e302421-82c7-42bb-a384-f41bca73d2eb
2019-05-02 14:54:34,021 INFO  org.apache.flink.runtime.blob.BlobServer                      - Started BLOB server at 0.0.0.0:33442 - max concurrent requests: 50 - max backlog: 1000
2019-05-02 14:54:34,044 INFO  org.apache.flink.runtime.metrics.MetricRegistryImpl           - No metrics reporter configured, no metrics will be exposed/reported.
2019-05-02 14:54:34,046 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - Trying to start actor system at localhost:0
2019-05-02 14:54:34,081 INFO  akka.event.slf4j.Slf4jLogger                                  - Slf4jLogger started
2019-05-02 14:54:34,105 INFO  akka.remote.Remoting                                          - Starting remoting
2019-05-02 14:54:34,128 INFO  akka.remote.Remoting                                          - Remoting started; listening on addresses :[akka.tcp://flink-metrics@localhost:34524]
2019-05-02 14:54:34,133 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - Actor system started at akka.tcp://flink-metrics@localhost:34524
2019-05-02 14:54:34,141 INFO  org.apache.flink.runtime.dispatcher.FileArchivedExecutionGraphStore  - Initializing FileArchivedExecutionGraphStore: Storage directory /tmp/executionGraphStore-a02136d4-c34e-4202-8e4f-68ab77812bcc, expiration time 3600000, maximum cache size 52428800 bytes.
2019-05-02 14:54:34,183 INFO  org.apache.flink.runtime.blob.TransientBlobCache              - Created BLOB cache storage directory /tmp/blobStore-e9345768-94e5-415a-a616-348365eaac2a
2019-05-02 14:54:34,270 INFO  org.apache.flink.configuration.Configuration                  - Config uses fallback configuration key 'jobmanager.rpc.address' instead of key 'rest.address'
2019-05-02 14:54:34,271 WARN  org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint    - Upload directory /tmp/flink-web-b0de0d0c-3882-4ccf-a9bc-e77c48cc3c49/flink-web-upload does not exist, or has been deleted externally. Previously uploaded files are no longer available.
2019-05-02 14:54:34,272 INFO  org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint    - Created directory /tmp/flink-web-b0de0d0c-3882-4ccf-a9bc-e77c48cc3c49/flink-web-upload for file uploads.
2019-05-02 14:54:34,276 INFO  org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint    - Starting rest endpoint.
2019-05-02 14:54:34,583 INFO  org.apache.flink.runtime.webmonitor.WebMonitorUtils           - Determined location of main cluster component log file: /mnt/installation/flink-1.8.0/log/flink-abhishek-standalonesession-4-abhishek-hp-probook-430-g3.log
2019-05-02 14:54:34,584 INFO  org.apache.flink.runtime.webmonitor.WebMonitorUtils           - Determined location of main cluster component stdout file: /mnt/installation/flink-1.8.0/log/flink-abhishek-standalonesession-4-abhishek-hp-probook-430-g3.out
2019-05-02 14:54:34,805 INFO  org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint    - Rest endpoint listening at localhost:8081
2019-05-02 14:54:34,806 INFO  org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint    - http://localhost:8081 was granted leadership with leaderSessionID=00000000-0000-0000-0000-000000000000
2019-05-02 14:54:34,806 INFO  org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint    - Web frontend listening at http://localhost:8081.
2019-05-02 14:54:34,929 INFO  org.apache.flink.runtime.rpc.akka.AkkaRpcService              - Starting RPC endpoint for org.apache.flink.runtime.resourcemanager.StandaloneResourceManager at akka://flink/user/resourcemanager .
2019-05-02 14:54:34,959 INFO  org.apache.flink.runtime.rpc.akka.AkkaRpcService              - Starting RPC endpoint for org.apache.flink.runtime.dispatcher.StandaloneDispatcher at akka://flink/user/dispatcher .
2019-05-02 14:54:34,997 INFO  org.apache.flink.runtime.resourcemanager.StandaloneResourceManager  - ResourceManager akka.tcp://flink@localhost:6123/user/resourcemanager was granted leadership with fencing token 00000000000000000000000000000000
2019-05-02 14:54:34,998 INFO  org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager  - Starting the SlotManager.
2019-05-02 14:54:35,018 INFO  org.apache.flink.runtime.dispatcher.StandaloneDispatcher      - Dispatcher akka.tcp://flink@localhost:6123/user/dispatcher was granted leadership with fencing token 00000000-0000-0000-0000-000000000000
2019-05-02 14:54:35,021 INFO  org.apache.flink.runtime.dispatcher.StandaloneDispatcher      - Recovering all persisted jobs.
2019-05-02 14:54:35,731 INFO  org.apache.flink.runtime.resourcemanager.StandaloneResourceManager  - Registering TaskManager with ResourceID 1b12a8033d8a788148dc343b538d847c (akka.tcp://[email protected]:45307/user/taskmanager_0) at ResourceManager
2019-05-02 14:55:22,480 INFO  org.apache.flink.runtime.dispatcher.StandaloneDispatcher      - Received JobGraph submission 6ccd855ac76bb63cb95103913c2efe95 (Flink Streaming Job).
2019-05-02 14:55:22,480 INFO  org.apache.flink.runtime.dispatcher.StandaloneDispatcher      - Submitting job 6ccd855ac76bb63cb95103913c2efe95 (Flink Streaming Job).
2019-05-02 14:55:22,535 INFO  org.apache.flink.runtime.rpc.akka.AkkaRpcService              - Starting RPC endpoint for org.apache.flink.runtime.jobmaster.JobMaster at akka://flink/user/jobmanager_0 .
2019-05-02 14:55:22,545 INFO  org.apache.flink.runtime.jobmaster.JobMaster                  - Initializing job Flink Streaming Job (6ccd855ac76bb63cb95103913c2efe95).
2019-05-02 14:55:22,554 INFO  org.apache.flink.runtime.jobmaster.JobMaster                  - Using restart strategy NoRestartStrategy for Flink Streaming Job (6ccd855ac76bb63cb95103913c2efe95).
2019-05-02 14:55:22,571 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Job recovers via failover strategy: full graph restart
2019-05-02 14:55:22,579 INFO  org.apache.flink.runtime.jobmaster.JobMaster                  - Running initialization on master for job Flink Streaming Job (6ccd855ac76bb63cb95103913c2efe95).
2019-05-02 14:55:22,579 INFO  org.apache.flink.runtime.jobmaster.JobMaster                  - Successfully ran initialization on master in 0 ms.
2019-05-02 14:55:22,595 INFO  org.apache.flink.runtime.jobmaster.JobMaster                  - No state backend has been configured, using default (Memory / JobManager) MemoryStateBackend (data in heap memory / checkpoints to JobManager) (checkpoints: 'null', savepoints: 'null', asynchronous: TRUE, maxStateSize: 5242880)
2019-05-02 14:55:22,604 INFO  org.apache.flink.runtime.jobmaster.JobManagerRunner           - JobManager runner for job Flink Streaming Job (6ccd855ac76bb63cb95103913c2efe95) was granted leadership with session id 00000000-0000-0000-0000-000000000000 at akka.tcp://flink@localhost:6123/user/jobmanager_0.
2019-05-02 14:55:22,606 INFO  org.apache.flink.runtime.jobmaster.JobMaster                  - Starting execution of job Flink Streaming Job (6ccd855ac76bb63cb95103913c2efe95) under job master id 00000000000000000000000000000000.
2019-05-02 14:55:22,607 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Job Flink Streaming Job (6ccd855ac76bb63cb95103913c2efe95) switched from state CREATED to RUNNING.
2019-05-02 14:55:22,615 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source: Custom Source (1/1) (98194ef84a5e0681a9db49772fa1e65a) switched from CREATED to SCHEDULED.
2019-05-02 14:55:22,631 INFO  org.apache.flink.runtime.jobmaster.slotpool.SlotPoolImpl      - Cannot serve slot request, no ResourceManager connected. Adding as pending request [SlotRequestId{d6bbb9779510f218ea06a212b8ddb3b3}]
2019-05-02 14:55:22,637 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Window(TumblingProcessingTimeWindows(5), ProcessingTimeTrigger, FoldFunction$1, PassThroughWindowFunction) -> Sink: Print to Std. Out (1/1) (b97895d8f20f026dc2e9417c44c86c30) switched from CREATED to SCHEDULED.
2019-05-02 14:55:22,640 INFO  org.apache.flink.runtime.jobmaster.JobMaster                  - Connecting to ResourceManager akka.tcp://flink@localhost:6123/user/resourcemanager(00000000000000000000000000000000)
2019-05-02 14:55:22,644 INFO  org.apache.flink.runtime.jobmaster.JobMaster                  - Resolved ResourceManager address, beginning registration
2019-05-02 14:55:22,644 INFO  org.apache.flink.runtime.jobmaster.JobMaster                  - Registration at ResourceManager attempt 1 (timeout=100ms)
2019-05-02 14:55:22,647 INFO  org.apache.flink.runtime.resourcemanager.StandaloneResourceManager  - Registering job manager [email protected]://flink@localhost:6123/user/jobmanager_0 for job 6ccd855ac76bb63cb95103913c2efe95.
2019-05-02 14:55:22,664 INFO  org.apache.flink.runtime.resourcemanager.StandaloneResourceManager  - Registered job manager [email protected]://flink@localhost:6123/user/jobmanager_0 for job 6ccd855ac76bb63cb95103913c2efe95.
2019-05-02 14:55:22,670 INFO  org.apache.flink.runtime.jobmaster.JobMaster                  - JobManager successfully registered at ResourceManager, leader id: 00000000000000000000000000000000.
2019-05-02 14:55:22,671 INFO  org.apache.flink.runtime.jobmaster.slotpool.SlotPoolImpl      - Requesting new slot [SlotRequestId{d6bbb9779510f218ea06a212b8ddb3b3}] and profile ResourceProfile{cpuCores=-1.0, heapMemoryInMB=-1, directMemoryInMB=0, nativeMemoryInMB=0, networkMemoryInMB=0} from resource manager.
2019-05-02 14:55:22,672 INFO  org.apache.flink.runtime.resourcemanager.StandaloneResourceManager  - Request slot with profile ResourceProfile{cpuCores=-1.0, heapMemoryInMB=-1, directMemoryInMB=0, nativeMemoryInMB=0, networkMemoryInMB=0} for job 6ccd855ac76bb63cb95103913c2efe95 with allocation id c831433fe33220e01eab5c62a219593a.
2019-05-02 14:55:22,746 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source: Custom Source (1/1) (98194ef84a5e0681a9db49772fa1e65a) switched from SCHEDULED to DEPLOYING.
2019-05-02 14:55:22,746 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Deploying Source: Custom Source (1/1) (attempt #0) to 1b12a8033d8a788148dc343b538d847c @ abhishek-hp-probook-430-g3 (dataPort=37667)
2019-05-02 14:55:22,763 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Window(TumblingProcessingTimeWindows(5), ProcessingTimeTrigger, FoldFunction$1, PassThroughWindowFunction) -> Sink: Print to Std. Out (1/1) (b97895d8f20f026dc2e9417c44c86c30) switched from SCHEDULED to DEPLOYING.
2019-05-02 14:55:22,764 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Deploying Window(TumblingProcessingTimeWindows(5), ProcessingTimeTrigger, FoldFunction$1, PassThroughWindowFunction) -> Sink: Print to Std. Out (1/1) (attempt #0) to 1b12a8033d8a788148dc343b538d847c @ abhishek-hp-probook-430-g3 (dataPort=37667)
2019-05-02 14:55:23,656 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Window(TumblingProcessingTimeWindows(5), ProcessingTimeTrigger, FoldFunction$1, PassThroughWindowFunction) -> Sink: Print to Std. Out (1/1) (b97895d8f20f026dc2e9417c44c86c30) switched from DEPLOYING to RUNNING.
2019-05-02 14:55:23,765 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source: Custom Source (1/1) (98194ef84a5e0681a9db49772fa1e65a) switched from DEPLOYING to FAILED.
org.apache.flink.streaming.runtime.tasks.StreamTaskException: Could not instantiate outputs in order.
    at org.apache.flink.streaming.api.graph.StreamConfig.getOutEdgesInOrder(StreamConfig.java:398)
    at org.apache.flink.streaming.runtime.tasks.StreamTask.createRecordWriters(StreamTask.java:1164)
    at org.apache.flink.streaming.runtime.tasks.StreamTask.<init>(StreamTask.java:212)
    at org.apache.flink.streaming.runtime.tasks.StreamTask.<init>(StreamTask.java:190)
    at org.apache.flink.streaming.runtime.tasks.SourceStreamTask.<init>(SourceStreamTask.java:51)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
    at org.apache.flink.runtime.taskmanager.Task.loadAndInstantiateInvokable(Task.java:1405)
    at org.apache.flink.runtime.taskmanager.Task.run(Task.java:689)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ClassNotFoundException: com.tp.flinktest.MyWikiAnalysis$2
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at org.apache.flink.runtime.execution.librarycache.FlinkUserCodeClassLoaders$ChildFirstClassLoader.loadClass(FlinkUserCodeClassLoaders.java:129)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:348)
    at org.apache.flink.util.InstantiationUtil$ClassLoaderObjectInputStream.resolveClass(InstantiationUtil.java:78)
    at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1620)
    at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1521)
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1781)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
    at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018)
    at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942)
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
    at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018)
    at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942)
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
    at java.io.ObjectInputStream.readObject(ObjectInputStream.java:373)
    at java.util.ArrayList.readObject(ArrayList.java:791)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1058)
    at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1909)
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
    at java.io.ObjectInputStream.readObject(ObjectInputStream.java:373)
    at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:566)
    at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:552)
    at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:540)
    at org.apache.flink.util.InstantiationUtil.readObjectFromConfig(InstantiationUtil.java:501)
    at org.apache.flink.streaming.api.graph.StreamConfig.getOutEdgesInOrder(StreamConfig.java:395)
    ... 11 more
2019-05-02 14:55:23,768 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Job Flink Streaming Job (6ccd855ac76bb63cb95103913c2efe95) switched from state RUNNING to FAILING.
org.apache.flink.streaming.runtime.tasks.StreamTaskException: Could not instantiate outputs in order.
    at org.apache.flink.streaming.api.graph.StreamConfig.getOutEdgesInOrder(StreamConfig.java:398)
    at org.apache.flink.streaming.runtime.tasks.StreamTask.createRecordWriters(StreamTask.java:1164)
    at org.apache.flink.streaming.runtime.tasks.StreamTask.<init>(StreamTask.java:212)
    at org.apache.flink.streaming.runtime.tasks.StreamTask.<init>(StreamTask.java:190)
    at org.apache.flink.streaming.runtime.tasks.SourceStreamTask.<init>(SourceStreamTask.java:51)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
    at org.apache.flink.runtime.taskmanager.Task.loadAndInstantiateInvokable(Task.java:1405)
    at org.apache.flink.runtime.taskmanager.Task.run(Task.java:689)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ClassNotFoundException: com.tp.flinktest.MyWikiAnalysis$2
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at org.apache.flink.runtime.execution.librarycache.FlinkUserCodeClassLoaders$ChildFirstClassLoader.loadClass(FlinkUserCodeClassLoaders.java:129)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:348)
    at org.apache.flink.util.InstantiationUtil$ClassLoaderObjectInputStream.resolveClass(InstantiationUtil.java:78)
    at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1620)
    at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1521)
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1781)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
    at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018)
    at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942)
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
    at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018)
    at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942)
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
    at java.io.ObjectInputStream.readObject(ObjectInputStream.java:373)
    at java.util.ArrayList.readObject(ArrayList.java:791)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1058)
    at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1909)
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
    at java.io.ObjectInputStream.readObject(ObjectInputStream.java:373)
    at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:566)
    at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:552)
    at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:540)
    at org.apache.flink.util.InstantiationUtil.readObjectFromConfig(InstantiationUtil.java:501)
    at org.apache.flink.streaming.api.graph.StreamConfig.getOutEdgesInOrder(StreamConfig.java:395)
    ... 11 more
like image 806
atmaish Avatar asked May 02 '19 13:05

atmaish

1 Answers

you have to post the pom.xml that you use to create your jar file. I already have the same issue and I added the following lines on my pom.xml. Check if you already have these parameters in yours:

<project xmlns="http://maven.apache.org/POM/4.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    ....
    <dependencies>
    ....
    </dependencies>
    <build>
        <plugins>
            <!-- Maven Shade Plugin -->
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-shade-plugin</artifactId>
                <version>3.2.0</version>
                <!-- Run shade goal on package phase -->
                <executions>
                    <execution>
                        <phase>package</phase>
                        <goals>
                            <goal>shade</goal>
                        </goals>
                        <configuration>
                            <!-- If you want to use ./bin/flink run <quickstart jar> uncomment 
                                the following lines. This will add a Main-Class entry to the manifest file -->
                            <transformers>
                                <transformer
                                    implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
                                    <mainClass>org.sense.flink.App</mainClass>
                                </transformer>
                                <!-- This bit merges the various GeoTools META-INF/services files -->
                                <transformer
                                    implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer" />
                            </transformers>
                            <createDependencyReducedPom>false</createDependencyReducedPom>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>
</project>
like image 146
Felipe Avatar answered Oct 20 '22 10:10

Felipe
Sign in to Comment

Related questions

Flink Custom Partition Function
Compose-Docker pull specific image:tag from a yml file service
Two questions on Flink externalized checkpoints
How to increase Flink taskmanager.numberOfTaskSlots to run it without Flink server(in IDE or fat jar)
Storage in Apache Flink
How to write the content of a Flink var to screen in Zeppelin?
Flink dynamic scaling
java.lang.NoSuchMethodException for init method in Scala case class
Test csv files equality with random line order (Junit)
Apache Flink: How to enable "upsert mode" for dynamic tables?
Flink window state size and state management
Apache Flink example job fails to run with "Job not found"
zipWithIndex on Apache Flink
Fllink Web UI not displaying records received in a Custom Source implementation
Flink exactly-once message processing
Flink Error - Key group is not in KeyGroupRange
What is Apache Flink's detached mode?
Apache Flink: My application does not resume from a checkpoint when I restart it
What do terms like Hash, Forward mean in the Flink plan?
Apache Flink Rest-Client Jar-Upload not working

Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!