Avoid socket inheritance when starting Linux service from C++ application

Question

I have a Linux service (daemon) that has multiple-threads and uses boost io_service listening on a TCP socket. When I receive a certain message on that socket I want to start another service with e.g. /etc/init.d/corosync start.

The problem is that after starting the service, when I quit my own service, the other service has inherited the sockets from my own service and it remains in a strange status where I cannot stop it the usual way.

Before exiting my process "MonitorSipServer" the open socket shows like this:

netstat -anop |grep 144
tcp        0      0 0.0.0.0:20144           0.0.0.0:*               LISTEN          4480/MonitorSipServ off (0.00/0/0)
tcp        0      0 140.0.24.181:20144      140.0.101.75:47036      ESTABLISHED 4480/MonitorSipServ off (0.00/0/0)

After exiting my process "MonitorSipServer" the open socket shows like this:

netstat -anop |grep 144
tcp        0      0 0.0.0.0:20144           0.0.0.0:*               LISTEN      4502/corosync    off (0.00/0/0)
tcp        0      0 140.0.24.181:20144      140.0.101.75:47036      ESTABLISHED 4502/corosync    off (0.00/0/0)

I have already tried with system, popen and with fork + execv or execve with null environment. It's always the same result or worse. My last hope was the Linux setsid command, but it has not worked either.

Any help would be appreciated. Regards, Jan

Answer 1

If you're referring to the socket descriptors themselves being inherited by exec'd child processes, and this being undesirable, then you could pass SOCK_CLOEXEC when creating the sockets with socket(2) to make sure that they are closed when you execute other programs. (This won't close the connection by the way, since your program still has a reference to the socket.)

If you are using some higher-level library, then check if there is some way to get it to pass this flag, or do fcntl(sock_fd, F_SETFD, fcntl(sock_fd, F_GETFD) | FD_CLOEXEC) to set the close-on-exec flag on the descriptor after it is created if you can get to it. (The fcntl(2) approach can be racy in a multi-threaded environment though, since some thread could exec(3) a program between the point where the socket is created and when FD_CLOEXEC is set on it.)

If the above won't work, then you could manually fork(2) and then close(2) the socket descriptors before executing a service. The advantage of SOCK_CLOEXEC is that the sockets will only be closed if the exec*() is actually successful, which sometimes makes it easier to recover from errors. In addition, SOCK_CLOEXEC can avoid some races and makes it harder to forget to close descriptors.

Answer 2

Boost.Asio supports the fork() system call:

• It requires the program to prepare and notify the io_service of the fork with io_service::notify_fork():

  io_service_.notify_fork(boost::asio::io_service::fork_prepare);
  if (fork() == 0)
  {
    io_service_.notify_fork(boost::asio::io_service::fork_child);
    ...
  }
  else
  {
    io_service_.notify_fork(boost::asio::io_service::fork_parent);
    ...
  }
  

  Failing to use this pattern results in unspecified behavior. For some configurations, the parent will fail to receive event notifications, as they are consumed by the child.

• It is the program's responsibility to handle any file descriptors accessible through Boost.Asio's public API. For example, if the parent has an open acceptor, then the child would need to explicitly invoke acceptor::close() before spawning its own children processes or replacing the process image. The fork support documentation states:

  Note that any file descriptors accessible via Boost.Asio's public API (e.g. the descriptors underlying basic_socket<>, posix::stream_descriptor, etc.) are not altered during a fork. It is the program's responsibility to manage these as required.

• Boost.Asio's fork support is not safe for multi-threaded processes. For multi-threaded processes, fork() states that the child may only invoke async-signal-safe operations between fork() and one of the exec() functions. io_service::notify_fork() does not make this guarantee, and the current implementation invokes non async-signal-safe operations.

With that said, I have seen applications not observe undesirable behavior when using Boost.Asio's fork() support in a multi-threaded process. However, if one wishes to meet the requirements of both fork() and Boost.Asio, then one solution is to fork the daemon process when it is still single threaded. The child process will remain single threaded and perform fork() and exec() when it is told to do so by the parent process via inter-process communication. This solution is suggested and demonstrated in this answer.