RabbitMQ + Memory Limits

Question

I'm just looking in to the config details of RabbitMQ and came across

[{rabbit, [{vm_memory_high_watermark, 0}, 
           {disk_free_limit, {mem_relative, 1.0}}
          ]
}]

What does this config mean?

vm_memory_high_watermark set to 0 means => Block all publishers immediately the rabbitmq app starts? But we still see rabbitmq able to queue whatever msgs we send.

16720 rabbitmq  20   0  142m  62m 2408 S    0  **1.6**   0:06.88 beam.smp

Whenever we send msgs to the broker we se this process' memory usage increasing. So, Does this mean the msgs are in memory although the watermark is set to 0?

We are curious to know what happens if the mem limit of ram reaches and still msgs are being sent? Either publishers are blocked? or The messages are swapped out to disk if available?

Answer 1

The vm_memory_high_watermark is a percentage value is related to memory flow control in RabbitMQ.

If you take a look at Memory flow control you will see that it says, under "Memory-Based Flow Control" heading:

The RabbitMQ server detects the total amount of RAM installed in the computer on startup and when rabbitmqctl set_vm_memory_high_watermark fraction is executed. By default, when the RabbitMQ server uses above 40% of the installed RAM, it raises a memory alarm and blocks all connections. Once the memory alarm has cleared (e.g. due to the server paging messages to disk or delivering them to clients) normal service resumes.

So by you setting this value to 0, then of course it will trigger straight away! If you want RabbitMQ to be allowed to use more memory then you will want to INCREASE the value.

Another important note:

The default memory threshold is set to 40% of installed RAM. Note that this does not prevent the RabbitMQ server from using more than 40%, it is merely the point at which publishers are throttled.

So if you try yo publish messages when the alarm has been raised then your publishers will be blocked from sending messages.

If you want to block all publishers then you would set the vm_memory_high_watermark to 0. If you want to 'disable' memory based flow control then set the vm_memory_high_watermark to 100. See details from above link:

A value of 0 makes the memory alarm go off immediately and thus disables all publishing (this may be useful if you wish to disable publishing globally; use rabbitmqctl set_vm_memory_high_watermark 0). To prevent the memory alarm from going off at all, set some high multiplier such as 100.