lost cassandra authentication user after adding a new node

Question

When ı add a new node to my cassandra cluster, ı lost my authentication user.

update seeds and restart nodes it turns default user -u cassandra -p cassandra

ı lost my user which ı have already created before

Answer 1

I have seen this happen before. What happens, is that the new node forces a token range re-calculation. If your new node is also a seed node, this complicates matters as seed nodes do not bootstrap data, and must be populated via repair/rebuild. Essentially, your pre-existing user is probably still there, but the node primarily responsible for its token in the system_auth.roles table has changed, but data movement has not occured.

First, double-check the replication strategy used on the system_auth keyspace. By default, it is set to {'class':'SimpleStrategy','replication_factor':'1'} which is not sufficient (IMO) for anything other than local development. I always recommend changing this to the NetworkTopologyStrategy, and then specify replication by data center.

With that complete, run a repair on each node:

nodetool repair system_auth -full

That should bring back your previous user.

Note: Instead of a full repair, you might be able to get away with querying each table in system_auth at consistency ALL (which forces a read repair):

dba@cqlsh> use system_auth;
dba@cqlsh:system_auth> consistency ALL;
Consistency level set to ALL.
dba@cqlsh:system_auth> SELECT COUNT(*) FROM roles;
dba@cqlsh:system_auth> SELECT COUNT(*) FROM role_permissions;
dba@cqlsh:system_auth> SELECT COUNT(*) FROM role_members;
dba@cqlsh:system_auth> SELECT COUNT(*) FROM resource_role_permissons_index;

With either the full repair or read repair complete, your previous user should work again.