Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Kafka: dynamically query configurations

Tags:

apache-kafka

Is there a way to access the configuration values in server.properties without direct access to that file itself?

I thought that:

kafka-configs.sh --describe --entity-type topics --zookeeper localhost:2181

might give me what I want, but I did not see the values set in server.properties. Just the following (I set 'ddos' as my own topic from kafka-topics.sh):

Configs for topics:ddos are
Configs for topics:__consumer_offsets are segment.bytes=104857600,cleanup.policy=compact

I was thinking I'd also see globally configured options, like this from the default configuration I have:

log.retention.hours=168

Thanks in advance.

like image 525
mpettis Avatar asked Mar 23 '16 20:03

mpettis


1 Answers

Since Kafka 0.11, you can use the AdminClient describeConfigs() API to retrieve configuration of brokers.

For example, skeleton code to retrieve configuration for broker 0:

Properties adminProps = new Properties();
adminProps.load(new FileInputStream("admin.properties"));
AdminClient admin = KafkaAdminClient.create(adminProps);

Collection<ConfigResource> resources = new ArrayList<>();
ConfigResource cr = new ConfigResource(Type.BROKER, "0");
resources.add(cr);
DescribeConfigsResult dcr = admin.describeConfigs(resources);
System.out.println(dcr.all().get());
like image 94
Mickael Maison Avatar answered Oct 11 '22 03:10

Mickael Maison