Boost POOL usage - singleton

Question

I have started using boost pools as singleton defined in boost/pool/singleton_pool.hpp as I need to allocate a lot of structures of same size repeatedly. The performance improvement is phenomenal as I was using malloc before.

The objects that I allocate are put in a list by a producer thread and a consumer thread takes these off the other end and frees the objects. But when I free object the memory usage of the process in task manager never reduces. I am guessing this is because a certain amount of memory is preallocated by the pool library?

Also when the data rate of the producer increases the total memory usage seems to increase in chunks ~ 10k but never reduces even after calling free for objects in the pool.

I would like to do some house keeping periodically to free the memory chunks to reduce the overall memory usage of the process. Is this possible? I cant use purge_memory as it will mean I would have to synchronise purge between produces and consumer. BTW does purge_memory frees the chunk i.e. reduce the memory usage in task manager?

I am programming in MS windows.

Thanks Niladri

PS - I have tried using release_memory by making the pool ordered ( ordered_malloc) but it always returns false.

UPDATE:

Have not tried purge_memory yet as the pool is shared between two threads. But have found that release_memory works but only for ordered pools and is slow in releasing memory as it only frees memory blocks with no allocations.

Purge will work faster I am sure.

Answer 1

According to the doc :

bool t.release_memory()

t must be ordered. Frees every memory block that doesn't have any allocated chunks. Returns true if at least one memory block was freed.

bool t.purge_memory()

Frees every memory block. This function invalidates any pointers previously returned by allocation functions of t. Returns true if at least one memory block was freed.

So to have your memory released, you need to use purge_memory(). You can do that only if every pointer you have got through the pool is no more used !

If release_memory returns true, it seems that your memory is still in use ...

my2c

Answer 2

This is how boost::pool works and this is why your allocations are so fast. When you de-allocate an object, you're not really de-allocating anything. boost::pool is basically saying that area of memory is available for another object of that size. So no, you won't see your memory-usage go down when you free objects, because that's how object-pooling works. You have a chunk of pre-allocated memory, and you use that chunk of memory to create your objects. A "pool of objects", as it were :)