Calling system() from multithreaded program

Question

We are working on a multithreaded memory-consuming application written in C++. We have to execute lots of shellscript/linux commands (and get the return code).

After reading that article we clearly understood that it would be a bad idea to use system() in our context.

A solution would be to fork after program starts and before creating any threads but the communication with that process may not be easy (socket, pipe ?).

The second solution we considered may consist of a dedicated deamon written in python (using xinetd ?) that would be able to process our system calls.

Have you ever had that problem? How did you solve it?

Note : Here is a more complete article that explain this problem : http://developers.sun.com/solaris/articles/subprocess/subprocess.html They recommend to use posix_spawn which uses vfork() instead of fork() (used in system()).

Answer 1

The article you link to mostly talks about issues if you fork() and don't immediately follow it with an exec*(). As system() typically would be implemented by a fork() followed by exec() most of the issues don't apply. One issue which does apply is the point about closing file descriptors, though; Unless you have specific reasons to do otherwise, opening files with O_CLOEXEC by default is probably a good rule of thumb.

One issue with fork()+exec() of large memory consuming applications is that if your OS is configured to not allow memory overcommit, the fork() may fail. One solution to this is to fork an "external process handler" process before you start allocating a lot of memory in your main process.

The best solution is if the functionality you require is available as a library, obviating the need to fork in the first place. That probably doesn't warm your heart in the short term, though.