RethinkDB: why does rethinkdb service use so much memory?

Question

After encountering situations where I found that rethinkdb service is down for unknown reason, I noticed it uses a lot of memory:

# free -m
                  total       used       free     shared    buffers     cached
    Mem:          7872       7744        128          0         30         68
    -/+ buffers/cache:       7645        226
    Swap:         4031        287       3744

# top
top - 23:12:51 up 7 days,  1:16,  3 users,  load average: 0.00, 0.00, 0.00
Tasks: 133 total,   1 running, 132 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.0%us,  0.2%sy,  0.0%ni, 99.8%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:   8061372k total,  7931724k used,   129648k free,    32752k buffers
Swap:  4128760k total,   294732k used,  3834028k free,    71260k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                           
 1835 root      20   0 7830m 7.2g 5480 S  1.0 94.1 292:43.38 rethinkdb                                                         
29417 root      20   0 15036 1256  944 R  0.3  0.0   0:00.05 top                                                               
    1 root      20   0 19364 1016  872 S  0.0  0.0   0:00.87 init



# cat log_file  | tail -9
2014-09-22T21:56:47.448701122 0.052935s info: Running rethinkdb 1.12.5 (GCC 4.4.7)...
2014-09-22T21:56:47.452809839 0.057044s info: Running on Linux 2.6.32-431.17.1.el6.x86_64 x86_64
2014-09-22T21:56:47.452969820 0.057204s info: Using cache size of 3327 MB
2014-09-22T21:56:47.453169285 0.057404s info: Loading data from directory /rethinkdb_data
2014-09-22T21:56:47.571843375 0.176078s info: Listening for intracluster connections on port 29015
2014-09-22T21:56:47.587691636 0.191926s info: Listening for client driver connections on port 28015
2014-09-22T21:56:47.587912507 0.192147s info: Listening for administrative HTTP connections on port 8080
2014-09-22T21:56:47.595163724 0.199398s info: Listening on addresses
2014-09-22T21:56:47.595167377 0.199401s info: Server ready

It seems a lot considering the size of the files:

# du -h
4.0K    ./tmp
156M    .

Do I need to configure a different cache size? Do you think it has something to do with finding the service surprisingly gone? I'm using v1.12.5

Answer 1

There were a few leak in the previous version, the main one being https://github.com/rethinkdb/rethinkdb/issues/2840

You should probably update RethinkDB -- the current version being 1.15. If you run 1.12, you need to export your data, but that should be the last time you need it since 1.14 introduced seamless migrations.

Answer 2

From Understanding RethinkDB memory requirements - RethinkDB

By default, RethinkDB automatically configures the cache size limit according to the formula (available_mem - 1024 MB) / 2. available_mem

You can change this via a config file as they document, or change it with a size (in MB) from the command line:

rethinkdb --cache-size 2048