Menu
I'm running a Cassandra cluster with version 2.2.3. The cluster consists of 3 nodes now, two of them are seeds and one is a normal node.
When I start repair on each node (command: nodetool repair -tr my_keyspace), I found the command was blocked on each node, I redirect the trace back information to the log and found there are many errors like the following:
Session completed with the following error: org.apache.cassandra.exceptions.RepairException: [repair #5717bb00-e685-11e5-801e-c71692f88562 on my_keyspace/node, (4856831381680181267,4878966233072304148]] Validation failed in /10.16.170.20
Has anyone faced the error before? Can we run Cassandra nodetool repair parallelly on each node?
348
Repair is a maintenance task which should be run on all then nodes once before each gc_grace_seconds period. For example default gc_grace_seconds is equal to 10 days so it is required to run repair on all the nodes once in this 10 day period.
Repair synchronizes the data between nodes by comparing their respective datasets for their common token ranges, and streaming the differences for any out of sync sections between the nodes. It compares the data with merkle trees, which are a hierarchy of hashes.
Scrub automatically discards broken data and removes any tombstoned rows that have exceeded gc_grace period of the table. If partition key values do not match the column data type, the partition is considered corrupt and the process automatically stops.
Compaction refers to a maintenance process in Cassandra, in which the SSTables are reorganized for data optimization of data structures on the disk.
I also found that if I execute nodetool repair without "-pr", the repair can complete successful more times, it seems that the "-pr" is not recommended after 2.2.x.:) Another I have found that execute nodetool repair without "-pr" parallelly on each node, it work good. However, there are no new data updated.
149
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
