Overloading fork( )

Question

I have overloaded the fork() system call and created my own version of fork() using RTLD_NEXT. That is, dlsym(RTLD_NEXT, fork). This will hit my version of fork. After this I want to replicate the task of actual fork() system call, that is, creating child process and returning the pid, and some more additional functionalities.

I am not able to figure out how to do that. I checked the kernel source code for fork() (fork.c) and could not figure out much.

Doing this:

dlsym(RTLD_NEXT,fork);  
int fork(void) {
    int pid=_fork(); // Trying to call actual fork does not work
    return pid;
}

How can I do that? Here is the link to kernel source code for fork: http://lxr.linux.no/linux+v2.6.32/kernel/fork.c#L10

Edit (pulled in from comments):

I am working on a leak detecting tool, and this tool detects a double free when a child process deletes the memory allocated by the parent. To overcome this i will override fork(), and whenever there is a fork(), the parent's memory allocation table will be duplicated to the child.

Answer 1

You aren't going to get anything useful from the kernel source code for fork. Your code will not be allowed to do the things the kernel does no matter what library trickery you manage. That's a hard boundary that cannot be breached without writing a kernel module.

All the library code for fork does is set things up and execute a special instruction that switches to kernel mode where the kernel's fork code executes. There is sort of a way to put this special instruction in your own code. It's the syscall function. Since fork takes no arguments, it should be relatively easy to use this function to make the system call.

But this is not what I recommend you do. I recommend you do this instead:

typedef int (*forkfunc_t)(void);

int fork(void)
{
     forkfunc_t sysfork = (forkfunc_t)dlsym(RTLD_DEFAULT, "fork");
     return sysfork();
}

Basically, whatever shared library hackery you do, you should basically find some way of retrieving the previous value of the fork function before you replace it with your own.

Answer 2

You should be able to call the actual fork with syscall(SYS_fork) after including <sys/syscall.h>. See syscall(2).