Docker memory limits not respected by mongo

Question

I am experiencing issue where mongodb does not respect imposed container memory limits and spills over into swap and slows to a crawl - the issue seems to be that mongo thinks all host memory is available to it. See https://github.com/dockerfile/mongodb/issues/34

My question is then - how does docker implement memory limits? I don't really have a big picture understanding of what the available options are for restricting resource access like this in unix - would someone be able to give a quick overview?

Thanks

Answer 1

There are a few points here.

First, any process running in your container will believe that it has access to as much memory as your host has (docker does not virtualize memory)

You can limit the memory your container is authorized to use by setting the parameter --memory. You can disable swap by setting --memory-swap to the exact same value as --memory:

If --memory and --memory-swap are set to the same value, this prevents containers from using any swap. This is because --memory-swap is the amount of combined memory and swap that can be used, while --memory is only the amount of physical memory that can be used.

Now, remember that even if you set --memory and --memory-swap, your application (mongoDB) will believe it has access to the total memory of your host system. In mongoDb 3.2, it will use 60% of your total memory, minus 1G. If you want to limit this, you need to configure storage.wiredTiger.engineConfig.cacheSizeGB

storage.wiredTiger.engineConfig.cacheSizeGB: The maximum size of the internal cache that WiredTiger will use for all data.

Check the mongoDB documentation here: https://docs.mongodb.com/v3.2/reference/configuration-options/#storage.wiredTiger.engineConfig.cacheSizeGB

And the docker documentation: https://docs.docker.com/config/containers/resource_constraints/#memory

Answer 2

In addition to what has already been specified in previous answers, Mongodb will respect the container memory setting from versions 3.6.13 and 4.0.9:

hostInfo.system.memLimitMB

The memory usage limit in megabytes.

For example, running in a container may impose memory limits that are lower than the total system memory. This memory limit, rather than the total system memory, is used as the maximum RAM available to calculate WiredTiger internal cache.

Available starting in MongoDB 4.0.9 (and 3.6.13)

https://docs.mongodb.com/manual/reference/command/hostInfo/#hostInfo.system.memLimitMB