GCC how to block system calls within a program?

Question

Does anyone tell me how to block some specific system calls within a program, please? I am building a system which takes a piece of C source code, compiles it with gcc and runs it. For security reasons, I need to prevent the compiled program from calling some system calls. Is there any way to do it, from the source code level (e.g. stripping the header files of gcc, detecting malicious external calls, ...) to the executable level?

Edited #1: Add details about malicious calls.

Edited #2: My system is a GNU/Linux one.

Edited #3:

I have tried some methods within a few days and here are the conclusions I've got so far:

1. Scanning the source code does not solve the main problem since one can always obsfucate his/her C source file quite well.
2. "Overriding C symbol" works well for libraries, but for system calls I have not achieved what I wanted. This idea is not dead, however, doing this would definitely cause me a lot of time hacking (gcc and/or ld).
3. Permission deescalation works like a charm. I could use fakeroot or a "guest" user to do it. This method is also the easiest to implement.

The other one is native client which I have not tried yet but I definitely would in near future due to the common between the project and my work.

Answer 1

As others have noted, it's impossible for a program to avoid making system calls, they permate the C library all over the place.

However you might be able to make some headway with careful use of the LD_PRELOAD mechanism, if your platform supports it (e.g. Linux): you write a shared library with the same symbol names as those in the C library, which are called instead of the intended libc functions. (For example, Electric Fence is built as a shared library on Debian-based systems and intercepts calls to malloc, free et al.)

I suspect you could use this mechanism to trap or argument-check calls to any libc functions you don't like, and perhaps to note those which you consider unconditionally safe. It might then be reasonable to scan the compiled executable for the code corresponding to INT 0x80 to trap out any attempts to make raw syscalls (0xcd 0x80 - though beware of false positives). However I have only give this a few moments of thought, I could easily have missed something or this might turn out to be impractical...

Answer 2

You could run the compiled program by forking it from a wrapper and use the Linux ptrace(2) facility to intercept and inspect all system calls invoked by the program.

The following example code shows a wrapper that runs the /usr/bin/w command, prints each system call invoked by the command, and terminates the command if it tries to invoke the write(2) system call.

#include <stdio.h>
#include <unistd.h>
#include <stdlib.h>
#include <sys/ptrace.h>
#include <sys/wait.h>
#include <sys/syscall.h>
#include <sys/reg.h>

#define BAD_SYSCALL __NR_write

int main(int argc, char *argv)
{
    pid_t child;
    int status, syscall_nr;

    child = fork();
    if (child == 0) {
        /* In child. */
        ptrace(PTRACE_TRACEME, 0, NULL, NULL);
        execl("/usr/bin/w", NULL, NULL);
        // not reached
    }

    /* In parent. */
    while (1) {
        wait(&status);

        /* Abort loop if child has exited. */
        if (WIFEXITED(status) || WIFSIGNALED(status))
            break;

        /* Obtain syscall number from the child's process context. */
        syscall_nr = ptrace(PTRACE_PEEKUSER, child, 4 * ORIG_EAX, NULL);
        printf("Child wants to execute system call %d: ", syscall_nr);

        if (syscall_nr != BAD_SYSCALL) {
            /* Allow system call. */
            printf("allowed.\n");
            ptrace(PTRACE_SYSCALL, child, NULL, NULL);
        } else {
            /* Terminate child. */
            printf("not allowed. Terminating child.\n");
            ptrace(PTRACE_KILL, child, NULL, NULL);
        }
    }

    exit(EXIT_SUCCESS);
}

You can do much more powerful things using ptrace, such as inspect and change a process' address space (e.g., to obtain and modify the parameters passed to a system call).

A good introduction can be found in this Linux Journal Article and its follow-up.