I have removed a regionserver from my HBase cluster.I removed its hostname from $HBASE_HOME/conf/regionservers
, and restarted the HBase cluster, but the HBase UI still shows the removed region server as a 'dead' region server.
The 'status' command in the hbase shell also shows it as a dead region server. How should I get rid of it ?
While talking about numbers, it can serve approximately 1,000 regions.
To get the region info about the table, you need to scan hbase:meta table. This command will give details of all the regions. Row key will have region name and there will be four column qualifiers.
So the solution is to go to WALs directory in HDFS (usually at /hbase/WALs) and remove the files of the old regionserver.
Absolute Yes, because Region server runs on the same servers as DataNodes.
Cluster getting haunted by dead regionserver :D
HBase may sometimes still show a decommissioned regionserver as dead. This is because, the WAL (Write-Ahead Log) of the dead regionserver was still in HDFS in the “splitting” state, so from HBase perspective it’s not dead !
So the solution is to go to WALs directory in HDFS (usually at /hbase/WALs) and remove the files of the old regionserver.
Found this at this wonderful blog kill zombie dead regionservers after much digging.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With