Getting total number of key-value pairs in RocksDB

Question

Is it possible to efficiently get the number of key-value pairs stored in a RocksDB key-value store?

I have looked through the wiki, and haven't seen anything discussing this topic thus far. Is such an operation even possible?

Answer 1

Codewisely, you could use db->GetProperty("rocksdb.estimate-num-keys", &num) to obtain the estimated number of keys stored in a rocksdb.

Another option is to use the sst_dump tool with --show_properties argument to get the number of entries, although the result would be per file basis. For example, the following command will show the properties of each SST file under the specified rocksdb directory:

sst_dump --file=/tmp/rocksdbtest-691931916/dbbench --show_properties --command=none

And here's the sample output:

Process /tmp/rocksdbtest-691931916/dbbench/000005.sst
Sst file format: block-based
Table Properties:
------------------------------
  # data blocks: 845
  # entries: 27857
  raw key size: 668568
  raw average key size: 24.000000
  raw value size: 2785700
  raw average value size: 100.000000
  data block size: 3381885
  index block size: 28473
  filter block size: 0
  (estimated) table size: 3410358
  filter policy name: N/A
  # deleted keys: 0
Process /tmp/rocksdbtest-691931916/dbbench/000008.sst
Sst file format: block-based
Table Properties:
------------------------------
  # data blocks: 845
  # entries: 27880
  raw key size: 669120
...

Combine with some shell commands, you will be able to get the total number of entries:

sst_dump --file=/tmp/rocksdbtest-691931916/dbbench --show_properties --command=none | grep entries | cut -c 14- | awk '{x+=$0}END{print "total number of entries: " x}'

And this will generate the following output:

total number of entries: 111507