Redis: How is "mem_fragmentation_ratio" impacting the performance?

Question

Here it is my question for you: How is "mem_fragmentation_rateo" impacting the performance in Redis?

I did some investigation in stackoverflow and in redis.io, but I couldn't really find a clear explanation about my question.

As also explained in this eBook https://www.datadoghq.com/wp-content/uploads/2013/09/Understanding-the-Top-5-Redis-Performance-Metrics.pdf at page #16, "The mem_fragmentation_ratio metric gives the ratio of memory used as seen by the operation system (used_memory_rss) to memory allocated by Redis (used_memory)."

So, as explained in http://redis.io/commands/INFO, "ideally, the used_memory_rss value should be only slightly higher than used_memory. When rss >> used, a large difference means there is memory fragmentation (internal or external), which can be evaluated by checking mem_fragmentation_ratio. When used >> rss, it means part of Redis memory has been swapped off by the operating system: expect some significant latencies".

So, when we have higher "used_memory" than "used_memory_rss", the negative impact on the Redis node is clear, because it means that Redis started to use Swap, and this is clearly not good for the performance.

But, what about a situation where we have an higher "used_memory_rss" than "used_memory"? This should probably mean that our memory allocator didn't "take back" part of the memory that assigned to Redis in the past, and that Redis is no longer using (maybe due to a temporary spike). However, I can't understand how this can negatively impact Redis performance. Why am I reading many many posts on internet where people are so concerned about an high "mem_fragmentation_ratio"? Also in the eBook that I mentioned before, they are suggesting to have a "mem_fragmentation_rateo" between 1 and 1.5.

In this situation, when we have an higher "used_memory_rss", is Redis really affected? Should we expect a degrade of Redis performance with an high "mem_fragmentation_ratio"?

I decided to ask you this question because I honestly think that, with an high "mem_fragmentation_ratio", we can have an issue "only" in the OS, that "may" suffer of a shortage of memory: the OS can't allocate new memory to new processes. Anyway, it shouldn't directly affect Redis (unless the OS starts to slowdown, and etc.).

Can you please clarify my doubts? Did you have any experience where you experienced high "mem_fragmentation_ratio", and you started to notice a degradation of the Redis' performance?

Thank you

Answer 1

I'm not a Redis guru, but from my experience and documentation I read, fragmentation_ratio doesn't affect performance directly. But it increases memory consumption. When your application adds new data and that data doesn't fill free fragments in memory (when bunches of new data are bigger than those fragments), that memory stays free. And thus you can run out of memory faster than you expect.