Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Kafka Producer Class Not Found Exception

I tried to implement a simple producer consumer example with kafka and I achieved with the following properties:

Properties configProperties = new Properties();
    configProperties.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG,"localhost:" + portNumber);
    configProperties.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG,"org.apache.kafka.common.serialization.ByteArraySerializer");
    configProperties.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG,"org.apache.kafka.common.serialization.StringSerializer");

    // Belirtilen property ayarlarına sahip kafka producer oluşturulur
    org.apache.kafka.clients.producer.Producer producer = new KafkaProducer(configProperties);

However when I try the exact same configs, and everything else the same, in another project which is a plugin for a data visualization software, I got this error:

.... // Here there is some other stuff but I thing the important one is the below one
Caused by: java.lang.NoClassDefFoundError: org/apache/kafka/clients/producer/Producer
at App.MyControlPanel.<init>(MyControlPanel.java:130)
at App.CytoVisProject.<init>(CytoVisProject.java:29)
... 96 more
Caused by: java.lang.ClassNotFoundException: org.apache.kafka.clients.producer.Producer
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 98 more

In the first one that I said it worked, I was using "mvn clean compile assembly:single", but in the second one I created a jar file for the whole project. Because the visualization software wants a jar file to install the plugin. Since every thing is same (At least I could not find any difference, I used same code) I guess the problem is about the way build the project. What happened here? What is the difference between "mvn clean compile assembly:single" and building a jar file in IntelliJ? Why I got this error and how to fix this? Thanks a lot for help!


As I said in the last comment of the first answer, I have a plugin which has manifest and transform as goal. Here:

<plugin>
            <groupId>com.springsource.bundlor</groupId>
            <artifactId>com.springsource.bundlor.maven</artifactId>
            <version>1.0.0.M2</version>
            <configuration>
                <outputManifest>C:\Users\USER\AppData\Local\Temp\archetype2tmp/META-INF/MANIFEST.MF</outputManifest>
                <failOnWarnings>false</failOnWarnings>
                <removeNullHeaders>true</removeNullHeaders>
                <manifestHeaders><![CDATA[Bundle-ManifestVersion: 2
Bundle-Name: CytoVisProject
Bundle-SymbolicName: CytoVisProject
Spring-DM-Version: ${pom.version}
]]></manifestHeaders>
            </configuration>
            <!-- generate the manifest automatically during packaging -->
            <executions>
                <execution>
                    <id>bundle-manifest</id>
                    <phase>package</phase>
                    <goals>
                        <goal>manifest</goal>
                        <goal>transform</goal>
                    </goals>
                </execution>
            </executions>
        </plugin>

If I use a shade plugin like below:

<plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-shade-plugin</artifactId>
            <version>3.1.0</version>
            <configuration>

            </configuration>
            <executions>
                <execution>
                    <phase>package</phase>
                    <goals>
                        <goal>shade</goal>
                    </goals>
                </execution>
            </executions>
        </plugin>

It does not works, because I need to use manifest and transform as goals in my plugin. How can I add kafka classes to the jar file that IntelliJ creates to solve this problem (I am not sure if this can solve or not)?

like image 753
JollyRoger Avatar asked Mar 13 '18 19:03

JollyRoger


1 Answers

I've found an easier solution. I have changed

kafkaProps.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG,"org.apache.kafka.common.serialization.StringSerializer");

to this

kafkaProps.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, StringSerializer.class);

And afterwards my code run, also as part of a pre-compiled plug-in.

like image 50
David Avatar answered Oct 24 '22 16:10

David