Is there a way to automatically close certain handles on a fork()?

Question

Background: I've got a large existing process (it happens to be on AIX, so basically POSIX semantics) that's part of an even larger system. The existing processes are designed to run continuously. A new requirement for this process is to handle a new kind of complex input stream. In order to reduce risk, I've decided to fork/exec a child process to do the actual input processing, which isolates the existing main process from problems such as crashes or hangs on malformed input data.

The child process reads data from stdin and writes to stdout after processing. I've got all the communication pipes set up so I can pass input data from the main process to the child, and read the output the other way, this all works fine (nonblocking to avoid deadlocks, etc). The child process lives as long as it takes for the main process to receive the (finite) input stream from an external source.

The question I have is about the pipe handles themselves. The main process informs the child that the input stream has completed by calling close() on the pipe attached to the child's stdin. This works as long as the only handle to the write end of that pipe is held by the main process. What if the main process, for some other unrelated reason, decides to fork? This would then create two handles to the write end of the pipe, meaning when I try to close the write end of the stdin pipe, the child won't notice because there is still another handle to the write end open. That other open handle is out of my control.

I know there is the FD_CLOEXEC bit that I can set on a file descriptor so that it is automatically closed when an exec() is done. However, this won't protect against the case where the main process forks but does not exec.

What's the general solution to this problem? I can think of only a couple of ideas:

1. Ensure (through inspection) that the existing process doesn't fork arbitrarily without also doing an exec. This may be possible but is not a general solution.
2. At startup, fork a long-lived helper process whose sole responsibility is to periodically fork/exec the child that does the actual processing. That way the helper's handle context is known and can be well-controlled. However, this is annoying because the helper needs some way to know that the input stream has ended other than closing its stdin.

Answer 1

Very few No standard system libraries fork without exec. It's quite unusual. There is no close-on-fork facility in Unix or Linux, and I doubt it on AIX. There are loadable kernel extensions on AIX if you really care that much. Presumably you know about fcntl for close on exec. If your co-workers are writing exec-less forks, then I can't offer much assistance.