Run an untrusted C program in a sandbox in Linux that prevents it from opening files, forking, etc.?

Question

I have used Systrace to sandbox untrusted programs both interactively and in automatic mode. It has a ptrace()-based backend which allows its use on a Linux system without special privileges, as well as a far faster and more poweful backend which requires patching the kernel.

It is also possible to create a sandbox on Unix-like systems using chroot(1), although that is not quite as easy or secure. Linux Containers and FreeBSD jails are a better alternative to chroot. Another alternative on Linux is to use a security framework like SELinux or AppArmor, which is what I would propose for production systems.

We would be able to help you more if you told as what exactly it is that you want to do.

EDIT:

Systrace would work for your case, but I think that something based on the Linux Security Model like AppArmor or SELinux is a more standard, and thus preferred, alternative, depending on your distribution.

EDIT 2:

While chroot(1) is available on most (all?) Unix-like systems, it has quite a few issues:

• It can be broken out of. If you are going to actually compile or run untrusted C programs on your system, you are especially vulnerable to this issue. And if your students are anything like mine, someone WILL try to break out of the jail.

• You have to create a full independent filesystem hierarchy with everything that is necessary for your task. You do not have to have a compiler in the chroot, but anything that is required to run the compiled programs should be included. While there are utilities that help with this, it's still not trivial.

• You have to maintain the chroot. Since it is independent, the chroot files will not be updated along with your distribution. You will have to either recreate the chroot regularly, or include the necessary update tools in it, which would essentially require that it be a full-blown Linux distribution. You will also have to keep system and user data (passwords, input files e.t.c.) synchronized with the host system.

• chroot() only protects the filesystem. It does not prevent a malicious program from opening network sockets or a badly-written one from sucking up every available resource.

The resource usage problem is common among all alternatives. Filesystem quotas will prevent programs from filling the disk. Proper ulimit (setrlimit() in C) settings can protect against memory overuse and any fork bombs, as well as put a stop to CPU hogs. nice(1) can lower the priority of those programs so that the computer can be used for any tasks that are deemed more important with no problem.

I wrote an overview of sandboxing techniques in Linux recently. I think your easiest approach would be to use Linux containers (lxc) if you dont mind about forking and so on, which don't really matter in this environment. You can give the process a read only root file system, an isolated loopback network connection, and you can still kill it easily and set memory limits etc.

Seccomp is going to be a bit difficult, as the code cannot even allocate memory.

Selinux is the other option, but I think it might be more work than a container.

Firejail is one of the most comprehensive tools to do that - it support seccomp, filesystem containers, capabilities and more:

https://firejail.wordpress.com/features-3/

You can use Qemu to test assignments quickly. This procedure below takes less than 5 seconds on my 5 year old laptop.

Let's assume the student has to develop a program that takes unsigned ints, each on their own line, until a line with "-1" arrives. The program should then average all the ints and output "Average: %f". Here's how you could test program completely isolated:

1. First, get root.bin from Jslinux, we'll use that as the userland (it has the tcc C-compiler):

   wget https://github.com/levskaya/jslinux-deobfuscated/raw/master/root.bin

2. We want to put the student's submission in root.bin, so set up the loop device:

   sudo losetup /dev/loop0 root.bin

   (you could use fuseext2 for this too, but it's not very stable. If it stabilizes, you won't need root for any of this)

3. Make an empty directory:

   mkdir mountpoint

4. Mount root.bin:

   sudo mount /dev/loop0 mountpoint

5. Enter the mounted filesystem:

   cd mountpoint.

6. Fix rights:

   sudo chown -R `whoami` .

7. mkdir -p etc/init.d

8. vi etc/init.d:

   #!/bin/sh
   cd /root
   echo READY 2>&1 > /dev/ttyS0
   tcc assignment.c 2>&1 > /dev/ttyS0
   ./a.out 2>&1 > /dev/ttyS0
   

9. chmod +x etc/init.d/rcS

10. Copy the submission to the VM:

    cp ~/student_assignment.c root/assignment.c

11. Exit the VM's root FS:

    cd ..

12. sudo umount mountpoint
13. Now the image is ready, we just need to run it. It will compile and run the submission after booting.
14. mkfifo /tmp/guest_output

15. Open a seperate terminal and start listening for guest output:

    dd if=/tmp/guest_output bs=1

16. In another terminal:

    qemu-system-i386 -kernel vmlinuz-3.5.0-27-generic -initrd root.bin -monitor stdio -nographic -serial pipe:/tmp/guestoutput (I just used the Ubuntu kernel here, but many kernels will work)

17. When the guest output shows "READY", you can send keys to the VM from the qemu prompt. For example, to test this assignment, you could do

    (qemu) sendkey 1
    (qemu) sendkey 4
    (qemu) sendkey ret
    (qemu) sendkey 1
    (qemu) sendkey 0
    (qemu) sendkey ret
    (qemu) sendkey minus
    (qemu) sendkey 1
    (qemu) sendkey ret
    

18. Now Average = 12.000000 should appear on the guest output pipe. If it doesn't, the student failed.

19. Quit qemu: quit

A program passing the test is here: https://stackoverflow.com/a/14424295/309483. Just use tcclib.h instead of stdio.h.

Try User-mode Linux. It has about 1% performance overhead for CPU-intensive jobs, but it may be 6 times slower for I/O-intensive jobs.

Running it inside a virtual machine should offer you all the security and restrictions you want.

QEMU would be a good fit for that and all the work (downloading the application, updating the disk image, starting QEMU, running the application inside it, and saving the output for later retrieval) could be scripted for automated tests runs.