I have done incremental data sync with help of kafka connect. Now i want to achieve same with custom query. But I am getting error.
My config file is
name=mysql-whitelist-timestamp-source
connector.class=io.confluent.connect.jdbc.JdbcSourceConnector
tasks.max=1
connection.url=jdbc:mysql://127.0.0.1:3306/demouser=root&password=root
query=select name from students3 where marks = 10
mode=timestamp table.whitelist=students3
timestamp.column.name=timestamp
topic.prefix=test-mysql-jdbc-
And getting below error:
ERROR WorkerConnector{id=mysql-whitelist-timestamp-source} Error while starting connector (org.apache.kafka.connect.runtime.WorkerConnector:119) org.apache.kafka.connect.errors.ConnectException: query may not be combined with whole-table copying settings.
To use Kafka Connect in standalone mode, we need to provide the mandatory parameters: Download a Kafka Connect connector, either from GitHub or Confluent Hub Confluent Hub. Create a configuration file for your connector. Use the connect-standalone.sh CLI to start the connector.
We shouldn't use the tag table.whitelist with the custom query. see the full explanation.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With