MSMQ consuming large amounts of memory when processing messages with NServiceBus

Question

I have two windows services which use NserviceBus. One writes messages to the queue and the other reads from it and do some processing. All the queues are transactional and the NserviceBus endpoints are configured as below.

.IsTransactional(true)
.IsolationLevel(IsolationLevel.ReadCommitted)
.MsmqTransport()
.RunTimeoutManager()
.UseInMemoryTimeoutPersister()
.MsmqSubscriptionStorage()
.DisableRavenInstall()
.JsonSerializer()

The issue is when a large amount of messages (170,000+) are queued, MSMQ service (mqsvc.exe) chews up quite a bit of memory (1.5 - 2.0 GB) and that memory doesn't get released for at least 5 - 6 hours. The average message size is around 5 - 10 KB. And it seems like the more messages you queue the more memory it uses. The NServiceBus based Windows Services memory consumption are in perfectly acceptable limits (50 - 100 MB) and do not increase no matter how many messages they process.

Any ideas on why MSMQ would use this much memory and takes quite long to release it? Thanks heaps.

Answer 1

This is perfectly normal. MSMQ uses storage in 4MB blocks of memory which map to the files in the Storage folder. 170,000 messages at 5-10kb each is 0.85-1.7GB so no surprise you're seeing so much virtual memory being allocated. To reduce the overhead of deleting and creating files as messages are removed or arrive, the storage files are kept for 6 hours. After this period, the empty files are deleted. You can configure this, as discussed in my blog post:

Forcing MSMQ to clean up its storage files