Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to create topic in Kafka with Python-kafka admin client?

is there any Python kafka admin client avilable to create topic/delete topic from python program ? I found some python apis but none of them have Admin api available ?

Does confluent have python admin api ?

like image 412
Bharat Avatar asked Oct 17 '22 19:10

Bharat


2 Answers

from kafka.admin import KafkaAdminClient, NewTopic
admin_client = KafkaAdminClient(bootstrap_servers="localhost:9092", client_id='test')

topic_list = []
topic_list.append(NewTopic(name="example_topic", num_partitions=1, replication_factor=1))
admin_client.create_topics(new_topics=topic_list, validate_only=False)
like image 73
Jubin Justifies Avatar answered Oct 21 '22 07:10

Jubin Justifies


I found this:

https://github.com/confluentinc/confluent-kafka-python

It worked for me, at least, at the topic creation, I specify one or multiple topics and it creates them in my kafka broker.

See my own answer:

Could I create a topic in Kafka via API?

I asked practically the same as you.

Example of use:

from confluent_kafka.admin import AdminClient, NewTopic

topic = sys.argv[1]

topics = ["newTopicExample","newTopicExample2"]

# Create topic in our Kafka, using kafka-python library.
a = AdminClient({'bootstrap.servers': 'myKafkaBrokerURL'})

new_topics = [NewTopic(topic, num_partitions=3, replication_factor=1) for topic in topics]

# Call create_topics to asynchronously create topics. A dict
# of <topic,future> is returned.
fs = a.create_topics(new_topics)

# Wait for each operation to finish.
for topic, f in fs.items():
    try:
        f.result()  # The result itself is None
        print("Topic {} created".format(topic))
    except Exception as e:
        print("Failed to create topic {}: {}".format(topic, e))
like image 22
icychocolate98 Avatar answered Oct 21 '22 08:10

icychocolate98