Apache Kafka Java Classes?

Question

I am trying to import kafka Classes with a Java Program, I couldn't find the Java Classes that I need to import, such as:

import kafka.message.Message;
import kafka.utils.Utils;

Could anyone tell me where I can find these Java Classes and how to provide them? It appears to me that Kafka is written in Scala and so I can't find those java classes after I download its source code.

Answer 1

You first need to build the Kafka release if you've not already done that with:

`tar xzf kafka-[VERSION].tgz`
`cd kafka-[VERSION]`
`./sbt update`
`./sbt package`

The simplest thing to do is then to just put the resulting kafka-[VERSION].jar (in kafka-0.7.2-incubating-src/core/target/scala_2.8.0/) and the other needed dependencies on your build path and it should work.

All dependencies (in kafka-[VERSION]-incubating-src/core/lib_managed/scala_2.8.0/compile/) are resolved with the sbt update.

You can follow the steps at http://kafka.apache.org/07/quickstart.html

Answer 2

Example minimal java program to use apache kafka jars, on Ubuntu:

The jars you need should be in the libs directory of your kafka installation. Copy them to where your java program can access them.

For newbs, a walkthrough. Put this in a file called Main.java:

import java.util.*;
import kafka.producer.*;
import kafka.utils.*;

public class Main{
   public static void main(String[] args) {
       Properties props = new Properties();
       props.put("metadata.broker.list", "broker1:9092,broker2:9092");
       props.put("serializer.class", "kafka.serializer.StringEncoder");
       props.put("partitioner.class", "example.producer.SimplePartitioner");
       props.put("request.required.acks", "1");
       ProducerConfig config = new ProducerConfig(props);
       System.out.println("k");
   }
}

Assuming you installed apache kafka correctly, then under your kafka installation directory you will find a libs directory. Inside there you will find a bunch of jar files.

The 3 jars I am interested in are:

kafka_2.10-0.8.1.1.jar
scala-library-2.10.1.jar
log4j-1.2.15.jar

Copy those jars to sit right next to your Main.java file.

Compile it:

javac -cp .:kafka_2.10-0.8.1.1.jar:scala-library-2.10.1.jar:log4j-1.2.15.jar Main.java

Run it:

el@apollo:~$ java -cp .:kafka_2.10-0.8.1.1.jar:scala-library-2.10.1.jar:log4j-1.2.15.jar Main
k
el@apollo:~$ 

Notes:

The program does not complain about the missing kafka.producer classes. Which means you imported the kafka classes correctly.