Why fork() works the way it does

Question

So, I have used fork() and I know what it does. As a beginner I was quite afraid of it (and I still don't understand it fully). The general description of fork() that you can find online is, that it copies the current process and assigns different PID, parent PID and the process will have different address space. All is good, however, given this functionality description a beginner would wonder "Why is this function so important... why would I want to copy my process?". So I did wonder and eventually I found out that's how you can call other processes from within your current process by means of the execve() family.

What I still don't understand is why do you have to do that this way? The most logical thing would be to have a function that you can call like

create_process("executable_path+name",params..., more params);  

which would spawn a new process and start running it at the beginning of main() and return the new PID.

What bothers me is the feeling that the fork/execve solution is doing potentially unneeded work. What if my process is using tons of memory? Does the kernel copy my page tables and such. I am sure it doesn't really allocate real memory unless I have touched it. Also, what happens if I have threads? It just seems to me that it's too messy.

Almost all description of what fork does, say it just copies the process and the new process starts running after the fork() call. This is indeed what happens but why does it happen this way and why is fork/execve the only way to spawn new processes and what is the most general unix way of creating a new process from your current one? Is there any other more effective way to spawn process?** Which wouldn't require to copy more memory.

This thread talks about the same issue, but I found it not quite satisfactory:

Thank you.

Answer 1

This is due to historical reasons. As explained at https://www.bell-labs.com/usr/dmr/www/hist.html, very early Unix did have neither fork() nor exec*(), and the way the shell executed commands was:

• Do the necessary initialization (opening stdin/stdout).
• Read a command line.
• Open the command, load some bootstrap code and jump to it.
• The bootstrap code read the opened command, (overwriting the shell's memory), and jumped to it.
• Once the command ended, it would call exit(), which then worked by reloading the shell (overwriting the command's memory), and jumping to it, going back to step 1.

From there, fork() was an easy addition (27 assembly lines), reusing the rest of the code.

In that stage of Unix development, executing a command became:

• Read a command line.
• fork() a child process, and wait for it (by sending a message to it).
• The child process loaded the command (overwriting the child's memory), and jumped to it.
• Once the command ended, it would call exit(), which was now simpler. It just cleaned its process entry, and gave up control.

Originally, fork() didn't do copy on write. Since this made fork() expensive, and fork() was often used to spawn new processes (so often was immediately followed by exec*()), an optimized version of fork() appeared: vfork() which shared the memory between parent and child. In those implementations of vfork() the parent would be suspended until the child exec*()'ed or _exit()'ed, thus relinquishing the parent's memory. Later, fork() was optimized to do copy on write, making copies of memory pages only when they started differing between parent and child. vfork() later saw renewed interest in ports to !MMU systems (e.g: if you have an ADSL router, it probably runs Linux on a !MMU MIPS CPU), which couldn't do the COW optimization, and moreover could not support fork()'ed processes efficiently.

Other source of inefficiencies in fork() is that it initially duplicates the address space (and page tables) of the parent, which may make running short programs from huge programs relatively slow, or may make the OS deny a fork() thinking there may not be enough memory for it (to workaround this one, you could increase your swap space, or change your OS's memory overcommit settings). As an anecdote, Java 7 uses vfork()/posix_spawn() to avoid these problems.

On the other hand, fork() makes creating several instances of a same process very efficient: e.g: a web server may have several identical processes serving different clients. Other platforms favour threads, because the cost of spawning a different process is much bigger than the cost of duplicating the current process, which can be just a little bigger than that of spawning a new thread. Which is unfortunate, since shared-everything threads are a magnet for errors.

Answer 2

Remember that fork was invented very early in Unix (& perhaps before) on machines which today seems ridiculously small (eg 64K bytes of memory).

And it is more in phase with the overall (original) philosophy of providing basic mechanisms, not policies, thru the most elementary possible actions.

fork just creates a new process, and the simplest way of thinking that is to clone the current process. So the fork semantics is very natural, and it is the simplest machanism possible.

Other system calls (execve) are in charge of loading a new executable, etc..

Separating them (and providing also pipe and dup2 syscalls) gives a lot of flexibility.

And on current systems, fork is implemented very efficiently (thru lazy copy on write pagination techniques). It is known that the fork mechanism makes Unix process creation quite fast (e.g. faster than on Windows or on VAX/VMS, which have system calls creating processes more similar to what you propose).

There is also the vfork syscall, which I don't bother using.

And the posix_spawn API is much more complex than fork or execve alone, so illustrates that fork is simpler...