Replace default STL allocator

Question

I have the source for a large (>250 files) library that makes heavy use of STL containers and strings. I need to run it in an embedded environment with limited heap, so I want to ensure that this library itself is limited in its heap usage.

The obvious solution is to create an allocator, but modifying the entire code base to include the allocator template parameter is a big job of last resort, and undesirable in case I ever want to take a new version of the source. Globally replacing new and delete is not feasible since that affects the entire image, not just this library.

My next thought was a stupid C macro trick, but that doesn't seem like it would be possible, although I admit to not being a clever macro author.

So I thought "is there a compiler or pragma switch to specify the allocator<> class at compile time"? But I'm open for anything.

The next question I'll ask, if anyone can come up with a solution, is how to do the same for new/delete within the set of files comprising this library.

I'm using the gcc 3.4.4 toolchain to run this under Cygwin, with a target of VxWorks, if that sparks any ideas.

Answer 1

I resorted to the preprocessor to get a possible solution, although it currently relies upon the GCC 3.4.4 implementation to work.

The GCC <memory> implementation includes the file <bits/allocator.h>, which in turn includes another file <bits/c++allocator.h>, which defines a macro that defines the class implementing the default allocator base class.

Since is found in a platform-dependent path (/lib/gcc/i686-pc-cygwin/3.4.4/include/c++/i686-pc-cygwin/bits), I don't feel (very) dirty in supplanting it with my own "platform-dependent" implementation.

So I just create a folder bits/ in the root of my source's include path, and then create the file c++allocator.h in that folder. I define the required macro to be the name of my allocator class and it works like a charm, since gcc searches my include paths prior to searching the system includes.

Thanks for all your responses. I think I can go with this "solution", which will only work as long as I'm using 3.4.4 probably.

Answer 2

You could benefit from using EASTL (Enterprise Arts STL (partial) implementation)

EASTL -- Electronic Arts Standard Template Library

This was intended for embedded/game development, in environments where global heap is really scarce, non-existent or problematic in general.

The allocator model of EASTL was inspired on (or resembles?) the ideas in the famous Towards a Better Allocator Model publication (PDF).

The EASTL lends itself well for custom allocators. In fact, it doesn't ship with an allocator, so providing (a minimal) one is required to even get you application to link.

Here is the github repo for EASTL: https://github.com/electronicarts/EASTL

Answer 3

So I thought "is there a compiler or pragma switch to specify the allocator<> class at compile time"? But I'm open for anything.

No there isn't.

Take a look here.

Allocators are a template argument in every stl container. You will need to change them. I have done the same thing in the past, when working on embedded. I could give you some pointers if you like :

Basic template allocator :

namespace PFM_MEM {
    template <class T>
    class CTestInstAllocator {
    public:
        // type definitions
        typedef size_t    size_type;
        typedef ptrdiff_t difference_type;
        typedef T*        pointer;
        typedef const T*  const_pointer;
        typedef T&        reference;
        typedef const T&  const_reference;
        typedef T         value_type;

        // rebind CTestInstAllocator to type U
        template <class U>
        struct rebind {
            typedef CTestInstAllocator<U> other;
        };

        // return address of values
        pointer address (reference value) const {
            return &value;
        }
        const_pointer address (const_reference value) const {
            return &value;
        }

        /* constructors and destructor
        * - nothing to do because the CTestInstAllocator has no state
        */
        CTestInstAllocator() {
        }
        CTestInstAllocator(const CTestInstAllocator&) {
        }
        template <class U>
        CTestInstAllocator (const CTestInstAllocator<U>&) {
        }
        ~CTestInstAllocator() {
        }

        // return maximum number of elements that can be allocated
        size_type max_size () const {
            return std::numeric_limits<size_t>::max() / sizeof(T);
        }

        // pvAllocate but don't initialize num elements of type T by using our own memory manager
        pointer allocate (size_type num) {
            /**
            * pvAllocate memory custom memory allocation scheme
            */
            return(pointer)(CPfmTestInstMemManager::pvAllocate(num*sizeof(T)));
        }
        // initialize elements of allocated storage p with value value
        void construct (pointer p, const T& value) {
            // initialize memory with placement new
            new((void*)p)T(value);
        }

        // destroy elements of initialized storage p
        void destroy (pointer p) {
            // destroy objects by calling their destructor
            p->~T();
        }
        // vDeallocate storage p of deleted elements
        void deallocate (pointer p, size_type num) {
            /**
            *Deallocate memory with custom memory deallocation scheme
            */
            CPfmTestInstMemManager::vDeallocate((void*)p);
        }
    };

    // return that all specializations of this CTestInstAllocator are interchangeable
    template <class T1, class T2>
    bool operator== (const CTestInstAllocator<T1>&,
        const CTestInstAllocator<T2>&) {
            return true;
    }
    template <class T1, class T2>
    bool operator!= (const CTestInstAllocator<T1>&,
        const CTestInstAllocator<T2>&) {
            return false;
    }
}

Take particular note at these lines :

/**
* pvAllocate memory custom memory allocation scheme
*/
return(pointer)(CPfmTestInstMemManager::pvAllocate(num*sizeof(T)));

// vDeallocate storage p of deleted elements
void deallocate (pointer p, size_type num) {
/**
*Deallocate memory with custom memory deallocation scheme
*/
CPfmTestInstMemManager::vDeallocate((void*)p);

Here is the place where you call your new and delete which work on your heap.

I could provide you with an example of how to construct some basic memory manager to help you further.