Capturing stdout of a Linux Process with Python using its PID

Question

I have a process in Linux running on the background that writes to stdout. I cannot to run it as a subprocess in a python script.

Is there any way to capture its standard out from from a python script using its PID? Sort of using strace -p<pid> -e writein bash.

Edits:

• I'm looking for something like python Queues or Pipes.
• I want to avoid changing the way the background process is called (e.g. forward the stdout to a pipe/file).
• Both the background process and the python script will be launched by the same user.

Answer 1

How to deal with this depends on some details, like exactly who owns the background process, and what you can change about how it's started.

Case 1: Not your background process

If you don't own the background process, you don't get to read its output, unless that process makes its output available somehow. (Imagine if a process could capture anyone else's process output... Someone else's script would be able to read the console output of your email reader or text editor.)

Case 2: Your background process

Even if your Python script will be run by the same user that runs the background process, it's pretty much too late for the caller to play with its file descriptors after it has started. You'd be much better off having that background process send its output wherever you wanted it in the first place --- either by redirecting its standard output, or by rewriting the process to log to a file.

Option: splitting standard output with tee

If something else depends on that background process's standard output, you can pipe the output through tee, which will save a copy of all its input in a file, and then repeats that input as standard output.

$ cat out.txt
cat: out.txt: No such file or directory
$ echo "This is a test." | tee out.txt | wc
      1       4      16
$ cat out.txt
This is a test.

Use tee -a <filename> to append to the file instead of overwrite it.

Option: /proc/ filesystem tricks

You could try reading from /proc/${bg_process_pid}/fd/1, as mentioned in Manuel Jacob's comment. My limited experience with /proc/ file descriptor symlinks is that they rarely do what you want, but I've only tried from the command line.

At best, I could get characters to show up in the right terminal window, but they weren't captured as output (nor seen as input to some child process). I'm not sure if this is bash or the /dev/pts/${some_number} pseudo-terminals interfering with what I was trying to do, some security feature in Linux, or me just not doing it right.

Case 3: A daemon process

If the background process is fully-fledged daemon process (doubtful, but some people include daemons when they say "background process"), it will have detached completely from its controlling terminal immediately after it started --- so it won't have a stdout in the usual sense at all. If you have permission to get a long listing of its /proc/${pid}/fd/0 and 1, you'll see they are symlinks to /dev/null.

I've spotted a few server processes that appear to redirect stderr to a file, but their system startup scripts don't do this. I suspect the open file descriptor 2 was actually opened by the daemon processes explicitly, and just happened to reuse the number 2.

Here's an example of an Apache httpd daemon on an oldish Fedora Linux server:

# ls -l /proc/1408/fd/[012]
lr-x------.  ...  /proc/1408/fd/0 -> /dev/null
l-wx------.  ...  /proc/1408/fd/1 -> /dev/null
l-wx------.  ...  /proc/1408/fd/2 -> /var/log/httpd/error_log

Samba's nmbd and smbd processes did something similar.