Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Kafka Connect - How to delete a connector

I created a cassandra-sink connector after that I made some changes in connector.properties file. After stopping the worker and starting it again, now when I add the connector using:

java -jar kafka-connect-cli-1.0.6-all.jar create cassandra-sink-orders < cassandra-sink-distributed-orders.properties

I get the following error:

Error: the Kafka Connect API returned: Connector cassandra-sink-orders already exists (409) 

How can I remove the existing connector?

like image 932
el323 Avatar asked Feb 23 '18 11:02

el323


People also ask

How do I delete connectors?

Either: Press Ctrl+Delete. Right-click on the connector and select the 'Delete Connector' option, or. Press the Delete key.

How do you stop a Kafka connector?

You can stop a specific connector by deleting the connector using the REST API [1]. You would need to make this REST call for every connector. If you have a lot of connectors running, you could write a little script that fetches the list of connectors [2] and deletes them one at a time, in a loop.

How do I restart Kafka connect connector?

There are two restart methods in the Kafka Connect REST API: POST /connectors/{connectorName}/restart – restarts only the Connector instance for the named connector. POST /connectors/{connectorName}/tasks/{taskNum}/restart – restarts only the Task instance for the named connector and specified task number.

What are connectors in Kafka connect?

The Kafka Connect JMS Source connector is used to move messages from any JMS-compliant broker into Apache Kafka®. The Kafka Connect Elasticsearch Service Sink connector moves data from Apache Kafka® to Elasticsearch. It writes data from a topic in Kafka to an index in Elasticsearch.


2 Answers

To delete a connector, you can run:

curl -X DELETE http://localhost:8083/connectors/<connector-name>
like image 183
foxwendy Avatar answered Sep 16 '22 22:09

foxwendy


You can use the Kafka Connect REST API, which includes an endpoint for DELETEing a connector.

curl -X DELETE http://$KAFKACONNECTWORKER_HOST:$KAFKACONNECTWORKER_PORT/connectors/$CONNECTOR_NAME

For example:

curl -X DELETE http://localhost:8083/connectors/src-jdbc-orders

🎥 See it in action here: https://www.youtube.com/watch?v=1EenWEm-5dg&t=378s

like image 38
Robin Moffatt Avatar answered Sep 19 '22 22:09

Robin Moffatt