Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Kafka-python get number of partitions for topic

I'm using: https://github.com/mumrah/kafka-python as a kafka api in Python. I want to get the number of partitions for a specified topic. How do I do that?

like image 341
user1413793 Avatar asked Apr 13 '15 18:04

user1413793


2 Answers

For those of you using Confluent-Python or the enterprise API. This can be done this way:

def count_partitions(my_partitions) -> int:
    count = 0
    for part in my_partitions:
        count = count + 1
    return count

cluster_data: ClusterMetadata = producer.list_topics(topic=TOPIC)
    topic_data: TopicMetadata = cluster_data.topics[TOPIC]
    available_partitions: PartitionMetadata = topic_data.partitions
    print(count_partitions(available_partitions))

like image 151
Drew Mash Avatar answered Sep 19 '22 16:09

Drew Mash


Might be a slightly simplistic solution, but:

from kafka import KafkaClient

client = KafkaClient('SERVER:PORT')
topic_partition_ids = client.get_partition_ids_for_topic(b'TOPIC')
len(topic_partition_ids)

tested on Python 3.4.3 / kafka-python 0.9.3

like image 28
Chaffelson Avatar answered Sep 21 '22 16:09

Chaffelson