From what Linux kernel/libc version is Java Runtime.exec() safe with regards to memory?

Question

At work one of our target platforms is a resource constrained mini-server running Linux (kernel 2.6.13, custom distribution based on an old Fedora Core). The application is written in Java (Sun JDK 1.6_04). The Linux OOM killer is configured to kill processes when memory usage exceeds 160MB. Even during high load our application never go over 120MB and together with some other native processes that are active we stay well within the OOM limit.

However, it turns out that the Java Runtime.getRuntime().exec() method, the canonical way to execute external processes from Java, has a particularly unfortunate implementation on Linux that causes spawned child processes to (temporarily) require the same amount of memory as the parent process since the address space is copied. The net result is that our application gets killed by the OOM killer as soon as we do Runtime.getRuntime().exec().

We currently work around this by having a separate native program do all external command execution and we communicate with that program over a socket. This is less than optimal.

After posting about this problem online I got some feedback indicating that this should not occur on "newer" versions of Linux since they implement the posix fork() method using copy-on-write, presumably meaning it will only copy pages that it needs to modify when it is required instead of the entire address space immediately.

My questions are:

• Is this true?
• Is this something in the kernel, the libc implementation or somewhere else entirely?
• From what version of the kernel/libc/whatever is copy-on-write for fork() available?

Answer 1

This is pretty much the way *nix (and linux) have worked since the dawn of time(or atleat the dawn of mmus).

To create a new process on *nixes you call fork(). fork() creates a copy of the calling process with all its memory mappings, file descriptors, etc. The memory mappings are done copy-on-write so (in optimal cases) no memory is actually copied, only the mappings. A following exec() call replaces the current memory mapping with that of the new executable. So, fork()/exec() is the way you create a new process and that's what the JVM uses.

The caveat is with huge processes on a busy system, the parent might continue to run for a little while before the child exec()'s causing a huge amount of memory to be copied cause of the copy-on-write. In VMs , memory can be moved around a lot to facilitate a garbage collector which produces even more copying.

The "workaround" is to do what you've already done, create an external lightweight process that takes care of spawning new processes - or use a more lightweight approach than fork/exec to spawn processes (Which linux does not have - and would anyway require a change in the jvm itself). Posix specifies the posix_spawn() function, which in theory can be implemented without copying the memory mapping of the calling process - but on linux it isn't.

Answer 2

Well, I personally doubt that this is true, since Linux's fork() is done via copy-on-write since God knows when (at least, 2.2.x kernels had it, and it was somewhere in the 199x).

Since OOM killer is believed to be a rather crude instrument which is known to misfire (f.e., it does not necessary kills the process that actually allocated most of the memory) and which should be used only as a last resport, it is not clear to me why you have it configured to fire on 160M.

If you want to impose a limit on memory allocation, then ulimit is your friend, not OOM.

My advice is to leave OOM alone (or disable it altogether), configure ulimits, and forget about this problem.