using glibc malloc hooks in a thread safe manner

Question

I would like to monitor the use of mallocs and frees in an application by using the malloc and free hooks.

Here's the documentation http://www.gnu.org/s/libc/manual/html_node/Hooks-for-Malloc.html

From the example page you can see that my_malloc_hook transiently switches the malloc hook off (or to the previous hook in the chain) before re-invoking malloc.

This is a problem when monitoring multi-threaded applications (see end of question for explanation).

Other examples of the use of malloc hook that I have found on the internet have the same problem.

Is there a way to re-write this function to work correctly in a multi-threaded application?

For instance, is there an internal libc function that the malloc hook can invoke that completes the allocation, without the need to deactivate my hook.

I can't look at the libc source code due to corporate legal policy, so the answer may be obvious.

My design spec says I cannot replace malloc with a different malloc design.

I can assume that no other hooks are in play.

---

UPDATE

Since the malloc hook is temporarily removed while servicing the malloc, another thread may call malloc and NOT get the hook.

It has been suggested that malloc has a big lock around it that prevents this from happening, but it's not documented, and the fact that I effectively recursively call malloc suggests any lock must either exist after the hook, or be jolly clever:

caller -> 
  malloc -> 
    malloc-hook (disables hook) -> 
      malloc -> # possible hazard starts here
        malloc_internals
      malloc <-
    malloc-hook (enables hook) <-
  malloc
caller

Answer 1

UPDATED

You are right to not trust __malloc_hooks; I have glanced at the code, and they are - staggeringly crazily - not thread safe.

Invoking the inherited hooks directly, rather than restoring and re-entering malloc, seems to be deviating from the the document you cite a little bit too much to feel comfortable suggesting.

From http://manpages.sgvulcan.com/malloc_hook.3.php:

Hook variables are not thread-safe so they are deprecated now. Programmers should instead preempt calls to the relevant functions by defining and exporting functions like "malloc" and "free".

The appropriate way to inject debug malloc/realloc/free functions is to provide your own library that exports your 'debug' versions of these functions, and then defers itself to the real ones. C linking is done in explicit order, so if two libraries offer the same function, the first specified is used. You can also inject your malloc at load-time on unix using the LD_PRELOAD mechanisms.

http://linux.die.net/man/3/efence describes Electric Fence, which details both these approaches.

You can use your own locking if in these debug functions if that is necessary.

Answer 2

I have the same problem. I have solved it with that example. If we do not define THREAD_SAFE, we have the example given by the man, and we have a segmentation error. If we define THREAD_SAFE, we have no segmentation error.

#include <malloc.h>
#include <pthread.h>

#define THREAD_SAFE
#undef  THREAD_SAFE

/** rqmalloc_hook_  */

static void* (*malloc_call)(size_t,const void*);

static void* rqmalloc_hook_(size_t taille,const void* appel)
{
void* memoire;

__malloc_hook=malloc_call; 
memoire=malloc(taille);    
#ifndef THREAD_SAFE
malloc_call=__malloc_hook;   
#endif
__malloc_hook=rqmalloc_hook_; 
return memoire;
}

/** rqfree_hook_ */   

static void  (*free_call)(void*,const void*);

static void rqfree_hook_(void* memoire,const void* appel)
{
__free_hook=free_call;   
free(memoire);            
#ifndef THREAD_SAFE
free_call=__free_hook;    
#endif
__free_hook=rqfree_hook_; 
}

/** rqrealloc_hook_ */

static void* (*realloc_call)(void*,size_t,const void*);

static void* rqrealloc_hook_(void* memoire,size_t taille,const void* appel)
{
__realloc_hook=realloc_call;     
memoire=realloc(memoire,taille); 
#ifndef THREAD_SAFE
realloc_call=__realloc_hook;    
#endif
__realloc_hook=rqrealloc_hook_; 
return memoire;
}

/** memory_init */

void memory_init(void)
{
  malloc_call  = __malloc_hook;
  __malloc_hook  = rqmalloc_hook_;

  free_call    = __free_hook;
  __free_hook    = rqfree_hook_;

  realloc_call = __realloc_hook;
  __realloc_hook = rqrealloc_hook_;
 }

 /** f1/f2 */

 void* f1(void* param)
 {
 void* m;
 while (1) {m=malloc(100); free(m);}
 }

 void* f2(void* param)
 {
 void* m;
 while (1) {m=malloc(100); free(m);}
 }

 /** main */
 int main(int argc, char *argv[])
 {
 memory_init();
 pthread_t t1,t2;

 pthread_create(&t1,NULL,f1,NULL);
 pthread_create(&t1,NULL,f2,NULL);
 sleep(60);
 return(0);
 }

Answer 3

Since all calls to malloc() will go through your hook, you can synchronize on a semaphore (wait until it is free, lock it, juggle the hooks and free the semaphore).

[EDIT] IANAL but ... If you can use glibc in your code, then you can look at the code (since it's LGPL, anyone using it must be allowed to have a copy of the source). So I'm not sure you understood the legal situation correctly or maybe you're not legally allowed to use glibc by your company.

[EDIT2] After some thinking, I guess that this part of the call path must be protected by a lock of some kind which glibc creates for you. Otherwise, using hooks in multi-threaded code would never work reliably and I'm sure the docs would mention this. Since malloc() must be thread safe, the hooks must be as well.

If you're still worried, I suggest to write a small test program with two threads which allocate and free memory in a loop. Increment a counter in the hook. After a million rounds, the counter should be exactly two million. If this holds, then the hook is protected by the malloc() lock as well.

[EDIT3] If the test fails, then, because of your legal situation, it's not possible to implement the monitor. Tell your boss and let him make a decision about it.

[EDIT4] Googling turned up this comment from a bug report:

The hooks are not thread-safe. Period. What are you trying to fix?

This is part of a discussion from March 2009 about a bug in libc/malloc/malloc.c which contains a fix. So maybe a version of glibc after this date works but there doesn't seem to be a guarantee. It also seems to depend on your version of GCC.

Answer 4

There is no way to use the malloc hooks in a thread-safe way while recursing into malloc. The interface is badly designed, probably beyond repair.

Even if you put a mutex in your hook code, the problem is that calls into malloc do not see those locks until after they have passed through the hook mechanism, and to pass through the hook mechanism, they look at global variables (the hook pointers) without acquiring your mutex. As you're saving, changing and restoring these pointers in one thread, allocator calls in another thread are affected by them.

The main design problem is that the hooks are null pointers by default. If the interface simply provided non-null default hooks which are the allocator proper (the bottom-level allocator which doesn't call any more hooks), then it would be simple and safe to add hooks: you could just save the previous hooks, and in the new hooks, recurse into malloc by calling the hold hooks, without fiddling with any global pointers (other than at hook installation time, which can be done prior to any threads start up).

Alternatively, glibc could provide an internal malloc interface which doesn't invoke the hooks.

Another sane design would be the use of thread-local storage for the hooks. Overriding and restoring a hook would be done in one thread, without disturbing the hooks seen by another thread.

As it stands, what you can do to use the glibc malloc hook safely is to avoid recursing into malloc. Do not change the hook pointers inside the hook callbacks, and simply call your own allocator.