Reduce RabbitMQ memory usage

Question

I'm trying to run RabbitMQ on a small VPS (512mb RAM) along with Nginx and a few other programs. I've been able to tweak the memory usage of everything else without difficulty, but I can't seem to get RabbitMQ to use any less RAM.

I think I need to reduce the number of threads Erlang uses for RabbitMQ, but I've not been able to get it to work. I've also tried setting the vm_memory_high_watermark to a few different values below the default (of 40%), even as low as 5%.

Part of the problem might be that the VPS provider (MediaTemple) allows me to go over my allocated memory, so when using free or top, it shows that the server has around 900mb.

Any suggestions to reduce memory usage by RabbitMQ, or limit the number of threads that Erlang will create? I believe Erlang is using 30 threads, based on the -A30 flag that I've seen on the process command.

Ideally I'd like RabbitMQ mem usage to be below 100mb.

Edit:

With vm_memory_high_watermark set to 5% (or 0.05 in the config file), the RabbitMQ logs report that RabbitMQ's memory limit is set to 51mb. I'm not sure where 51mb is coming from. Current VPS allocated memory is 924mb, so 5% of that should be around 46mb.

According to htop/free before starting up RabbitMQ, I'm sitting around 453mb of used ram, and after start RabbitMQ I'm around 650mb. Nearly 200mb increase. Could it be that 200mb is the lower limit that RabbitMQ will run with?

Edit 2

Here are some screenshots of ps aux and free before and after starting RabbitMQ and a graph showing the memory spike when RabbitMQ is started.

Edit 3

I also checked with no plugins enabled, and it made very little difference. It seems the plugins I had (management and its prerequisites) only added about 8mb of ram usage.

Edit 4

I no longer have this server to test with, however, there is a conf setting delegate_count that is set to a default of 16. As far as I know, this spawns 16 sup-procs for rabbitmq. Lowering this number on smaller servers may help reduce the memory footprint. No idea if this actually works, or how it impacts performance, but it's something to try.

Answer 1

The appropriate way to limit memory usage in RabbitMQ is using the vm_memory_high_watermark. You said:

I've also tried setting the vm_memory_high_watermark to a few different values below the default (of 40%), even as low as 5%.

This should work, but it might not be behaving the way you expect. In the logs, you'll find a line that tells you what the absolute memory limit is, something like this:

=INFO REPORT==== 29-Oct-2009::15:43:27 ===
Memory limit set to 2048MB.

You need to tweak the memory limit as needed - Rabbit might be seeing your system as having a lot more RAM than you think it has if you're running on a VPS environment.

Sometimes, Rabbit can't tell what system you're on and uses 1GB as the base point (so you get a limit of 410MB by default).

Also, make sure you are running on a version of RabbitMQ that supports the vm_memory_high_watermark setting - ideally you should run with the latest stable release.

Answer 2

Make sure to set an appropriate QoS prefetch value. By default, if there's a client, the Rabbit server will send any messages it has for that client's queue to the client. This results in extensive memory usage both on the client & the server.

Drop the prefetch limit down to something reasonable, like say 100, and Rabbit will keep the remaining messages on disk on the server until the client is really ready to process them, and your memory usage will go way way down on both the client & the server.

Note that the suggestion of 100 is just a reasonable place to start - it sure beats infinity. To really optimize that number, you'll want to take into consideration the messages/sec your client is able to process, the latency of your network, and also how large each of your messages is on average.