Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Kafka Streams closing processor's state store

JavaDoc description for Processor.close() in Kafka 1.0.1 says, that:

Note: Do not close any streams managed resources, like StateStores here, as they are managed by the library.

In addition, the JavaDoc description of StateStore.close() says, that:

Users only need to implement this function but should NEVER need to call this api explicitly as it will be called by the library automatically when necessary

But I've found an example in the documentation, in which a state store is being explicitly closed inside that method:

@Override
public void close() {
    // close the key-value store
    kvStore.close();
}

So, I'm kind of lost. Should I close the state store inside a Processor, or not?

like image 651
Dth Avatar asked Mar 08 '18 19:03

Dth


People also ask

What is state store in Kafka Streams?

State. Kafka Streams provides so-called state stores, which can be used by stream processing applications to store and query data, which is an important capability when implementing stateful operations.

How do I close a Kafka stream?

To stop the application instance, call the KafkaStreams#close() method: // Stop the Kafka Streams threads streams. close();

Are Kafka Streams persistent?

Kafka Streams are backed by a persistent or in-memory state store, themselves being backed by Kafka changelog topics, providing full fault tolerance. The accompanying Kafka Streams Spring Boot application source code is available here.


1 Answers

No, you should not close the store.

The example code snippet is wrong. Thanks for pointing it out. I opened a PR to fix it: https://github.com/apache/kafka/pull/4667

like image 88
Matthias J. Sax Avatar answered Sep 30 '22 00:09

Matthias J. Sax