Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Is there a malloc implementation which does bookkeeping outside its own heap?

I need to manage a memory heap, with the constraint that this memory should only be written to, never read, i.e. the malloc implementation should keep the bookkeeping information separately from the heap it manages, on the normal heap, and should in fact never touch the specific heap it manages. I was hoping to use a tested, optimized, off the shelf solution for that, if one is available. Examples of use include OpenGL VBOs and memory on external units of embedded systems.

I glanced at dlmalloc, and from the documentation, it seems to tag the memory blocks it allocates from both sides with bookkeeping information. Googling didn't do any good either - perhaps i don't have the right keywords to find what i'm looking for.

Clarifications: as a separate heap, i mean what i define to be a heap. I want to tightly use memory with small allocations within one or a small number of pre-allocated blocks. I don't even care if the bookkeeping information (outside the thus managed heap) is larger than the data inside :) Furthermore, the application itself will use stock malloc and heap for its operation, and only use those blocks for special purpose, which boils down to memory regions for speaking to external hardware, where writes from application are the purpose, reads are impossible or expensive. This is not a general malloc question, i was merely hoping to leverage something where a lot of research and testing has gone into.

like image 571
3yE Avatar asked Dec 17 '10 00:12

3yE


People also ask

Where is malloc implemented?

The malloc implementation in the GNU C Library is derived from ptmalloc (pthreads malloc), which in turn is derived from dlmalloc (Doug Lea malloc). This malloc may allocate memory in two different ways depending on their size and certain parameters that may be controlled by users.

How does malloc and free work internally?

How malloc() and free() works depends on the runtime library used. Generally, malloc() allocates a heap (a block of memory) from the operating system. Each request to malloc() then allocates a small chunk of this memory be returning a pointer to the caller.

How malloc is implemented in Linux?

When one calls malloc , memory is taken from the large heap cell, which is returned by malloc . The rest is formed into a new heap cell that consists of all the rest of the memory. When one frees memory, the heap cell is added to the end of the heap's free list.

What does malloc do in c?

What is malloc() in C? malloc() is a library function that allows C to allocate memory dynamically from the heap. The heap is an area of memory where something is stored. malloc() is part of stdlib. h and to be able to use it you need to use #include <stdlib.


2 Answers

and should in fact never touch the specific heap it manages.

What if it does not manage the heap? See this malloc function utilizing a particular implementation that neither manages the [heap] area (cf. /proc/$$/maps), nor stores its metadata in adressable memory, and yet, gives your program unique adressable memory.

void *mymalloc(size_t len)
{
        void *x = mmap(NULL, len, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
        return (x == (void *)-1) ? NULL : x;
}

And now for the killer revelation: glibc uses exactly that for sufficiently large allocations.

like image 152
user502515 Avatar answered Oct 03 '22 16:10

user502515


It's not a ready to use library, but the resource management code in the Linux kernel does exactly this to manage resources such as PCI address space.

like image 34
Adrian Cox Avatar answered Oct 03 '22 16:10

Adrian Cox