Kafka Source in Spring Cloud Data Flow

Question

I am migrating from Spring XD to Spring Cloud Data Flow. When I am looking for module list I realised that some of the sources are not listed in Spring Cloud Flow - One of them is KAFKA source.

My question is why KAFKA source is removed from standard sources list in spring cloud data flow ?

Answer 1

When I am looking for module list I realised that some of the sources are not listed in Spring Cloud Flow

Majority of the applications are ported over and the remaining are incrementally prioritized - you can keep track of the remaining subset in the backlog.

My question is why KAFKA source is removed from standard sources list in spring cloud data flow ?

Kafka is not removed and in fact, we are highly opinionated about Kafka in the context of streaming use-cases so much so that it is baked into the DSL directly. More details here.

For instance,

(i) if you've to consume from a Kafka topic (as a source), your stream definition would be:

stream create --definition ":someAwesomeTopic > log" --name subscribe_to_broker --deploy

(ii) if you've to write to a Kafka topic (as a sink), your stream definition would be:

stream create --definition "http --server.port=9001 > :someAwesomeTopic" --name publish_to_broker --deploy

(where *someAwesomeTopic* is the named destination, a topic name)