Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Kafka connect Debezium Postgres Cloud SQL

I am trying to connect to Cloud SQL postgres using Debezium Postgres connector. And I am getting the following error in logs.

connect_1    | org.apache.kafka.connect.errors.ConnectException: org.postgresql.util.PSQLException: FATAL: must be superuser or replication role to start walsender
connect_1    |  at io.debezium.connector.postgresql.PostgresConnectorTask.start(PostgresConnectorTask.java:127)

It occurred to me that I need to add REPLICATION to my user role. However, CloudSQL docs say the following.

When you create a new Cloud SQL for PostgreSQL instance, the default postgres user is already created for you, though you must set its password. The postgres user is part of the cloudsqlsuperuser role, and has the following attributes (privileges): CREATEROLE, CREATEDB, and LOGIN. It does not have the SUPERUSER or REPLICATION attributes.

Now because of this, I can't ALTER ROLE to add REPLICATION

ps: the cloud SQL instance is configured for high availability.

like image 540
Amit Basuri Avatar asked Oct 15 '22 10:10

Amit Basuri


1 Answers

Debezium requires the Postgres wal2json extension, which is not currently supported by Cloud SQL.There is an ongoing feature request for this which you can check. You will need to setup Postgresql on GCE and setup Debezium there.

If this helps, do accept the answer

like image 141
Jayadeep Jayaraman Avatar answered Nov 02 '22 09:11

Jayadeep Jayaraman