How to get notified when a process ends under linux?

Question

How do I monitor a number of processes and if one process ends, I would like to run some code.

I have found several examples which used polling to achieve this but I am looking for a method to get pushed (probably by the OS) when a process dies. Is this possible with C or C++? It should run on any modern Linux.

If there is any chance, I would like to do that without needing root privileges.

EDIT:

The job of this whole program is to monitor these processes and send this information to another server where it gets integrated into a website.

I have not started these processes but I could ensure that they are started as the same user.

I thought it should be possible because the top / ps command under Linux also gives you information about processes you haven't started.

Answer 1

In general on Linux you can't be notified (with waitpid(2) or SIGCHLD -see signal(7)- ...) about non-child processes, or processes outside of your process group or session.

And on some Linux your (non-root) process might not even be allowed to query the existence of other processes.

Of course, there is /proc/ (having one numerical-named directory per process, e.g. /proc/1234/ for process of pid 1234, see proc(5)) that you might regularly scan (e.g. with readdir(3)...) but you cannot be notified (see inotify(7), which does not work for pseudo-file systems like /proc/ ...) about changes inside it. Notice that /proc/ is a pseudo file system, and accessing it does not involve any disk IO so is quite quick.

So what you could do is scan /proc/ every few seconds using opendir(3), readdir, closedir(3), sleep(3) in a loop. BTW, that in theory is not fail-proof (in principle, not in practice, the kernel might reuse the same pid within a few seconds), and probably won't catch all short-living processes (such as ls shell commands).

Such a periodic scan of /proc is very probably what the top(1) utility is doing. You could check that by diving into the source code of top or by strace(1)-ing it.

If your C code knows already the pid of some process and simply wants to check the existence of that process, it can use kill(2) with a signal number 0.

See also systemd & credentials(7).

If you can change the code of the monitored programs or replace them (e.g. by your small C program wrapping them) things are very different; e.g. you could replace /usr/bin/foo with /usr/local/bin/foo-wrapper and code a foo-wrapper.c which fork-s & exec-s the original /usr/bin/foo then waitpid(2) on it and finally send(2) or write(2) some message on some socket(7) or fifo(7) or pipe(7), and use a poll(2) based event loop in your monitor. If you can get all the programs fork-ed by your monitor things are also different (use waitpid...). See my execicar.c program for inspiration.

Answer 2

You can configure auditd daemon to create audit records (log lines) when a process ends. And then monitor auditd log file with inotify.

Provided you have access to auditd configuration and its log file.