Kafka Connector to IBM DB2

Question

I'm currently working in a Mainframe Technology where we store the data in IBM DB2.

We got a new requirement to use scalable process to migrate the data to a new messaging platform including new database. For that we have identified Kafka is a suitable solution with either KSQLDB or MONGODB.

Can someone able to tell or direct me on how can we connect to IBM DB2 from Kafka to import the data and place it in either KSQLDB or MONGODB?

Any help is much appreciated.

Answer 1

To import the data from IBM DB2 into Kafka, You need to use any connector like the Debezium connector for DB2.

The information regarding the connector can be found in the following. https://debezium.io/documentation/reference/connectors/db2.html

Connector Configuration

You can also use JDBC Source Connector for the same functionality. The following links are helpful for the configuration.

https://www.confluent.io/blog/kafka-connect-deep-dive-jdbc-source-connector/

A Simple diagram for events flows from RDMS to Kafka topic.

After placing the data into Kafka, we need to transfer that data MongoDb. We need to use Mongo Db Connector to transfer the data from Kafka to mongo Db.

https://www.mongodb.com/blog/post/getting-started-with-the-mongodb-connector-for-apache-kafka-and-mongodb-atlas

https://www.confluent.io/hub/mongodb/kafka-connect-mongodb