mprotect entire program, to run dangerous code

Question

I have a small program that mmaps potentially dangerous executable code (with PROT_EXEC), calls prctl(PR_SET_SECCOMP, 1) and then executes this mmap'd code. This is all well and good, and allows me to "save" the state of the evaluation by sync the mmap'd region to disk, and reload it later (most likely on another machine for load balancing). However, this technique doesn't always work -- because this code might have made changes to the program that are not in the mmap'd region, and this information will be lost.

So what I would like to do, is make absolutely everything (other than this mmap'd region) read-only before calling the code. This way I have a guarantee that the executable code can't change the state of anything other than the mmap'd region which I can serialize/deserialize at will.

BTW this is Linux on x86_64

Thanks

Answer 1

Firstly, an observation: There's nothing that says you have to mmap() to get machine instructions into memory or to save them back to a file. read() and write() can do this too, just note that you should make a writable and executable private mapping for this purpose.

Obviously you can't reliably disable writing to the area of the stack that will be calling into the executable code that you'll load, if it's to be executed within the same process since this will render the stack unusable. You might work around this by annotation your variables or using assembly.

Your next option is to fork(). You could exec in the child into a special wrapper executable that allows for minimal damage and introspection by malicious executable code (provides simply load/dump), or you could do the same by having the child modify itself to the same effect. This still isn't 100% safe.

Proposal0

• Create a stand alone binary that is linked against minimal libraries (-nodefaultlibs).
• After a fork, ptrace(PTRACE_TRACEME) in the child (so that you can read memory contents reliably and do other interventions), and close all handles except that of a pipe (just in stdin for simplicity). exec() into the aforementioned wrapper binary.

In the wrapper binary:

• mmap a private region at a known location with write and execute permissions. Alternatively you can statically allocate this region if the size is fixed.
• Read the contents of the pipe into the region.
• Close the pipe. Now the process has no open handles.
• prctl(PR_SET_SECCOMP, 1). Now the only valid system calls are _exit and sigreturn. Since the process can't raise, sigreturn should have no useful effect.
• Remove write permissions from the main stack (should be the only stack). Since you have no intention of returning, and will jump immediately afterward, you shouldn't need to touch the stack again.
• Jump to the starting location inside the region. Do this using assembly, or create a function pointer and invoke it (if you can get it to work without pushing to the stack). Now you should be executing a region of memory that is the only writable region available. The main stack was protected, and the heap should not be in use due to lack of library support.

In the parent:

• Using ptrace or wait, catch erroneous or successful completion.
• Read the mapped region at the known location via /proc/<pid>/mem or equivalent to file.