UNIX named PIPE end of file

Question

I'm trying to use a unix named pipe to output statistics of a running service. I intend to provide a similar interface as /proc where one can see live stats by catting a file.

I'm using a code similar to this in my python code:

while True:
  f = open('/tmp/readstatshere', 'w')
  f.write('some interesting stats\n')
  f.close()

/tmp/readstatshere is a named pipe created by mknod.

I then cat it to see the stats:

$ cat /tmp/readstatshere
some interesting stats

It works fine most of the time. However, if I cat the entry several times in quick successions, sometimes I get multiple lines of some interesting stats instead of one. Once or twice, it has even gone into an infinite loop printing that line forever until I killed it. The only fix that I've got so far is to put a delay of let's say 500ms after f.close() to prevent this issue.

I'd like to know why exactly this happens and if there is a better way of dealing with it.

Thanks in advance

Answer 1

A pipe is simply the wrong solution here. If you want to present a consistent snapshot of the internal state of your process, write that to a temporary file and then rename it to the "public" name. This will prevent all issues that can arise from other processes reading the state while you're updating it. Also, do NOT do that in a busy loop, but ideally in a thread that sleeps for at least one second between updates.

Answer 2

What about a UNIX socket instead of a pipe?

In this case, you can react on each connect by providing fresh data just in time.

The only downside is that you cannot cat the data; you'll have to create a new socket handle and connect() to the socket file.

MYSOCKETFILE = '/tmp/mysocket'
import socket
import os
try:
    os.unlink(MYSOCKETFILE)
except OSError: pass
s = socket.socket(socket.AF_UNIX)
s.bind(MYSOCKETFILE)
s.listen(10)
while True:
    s2, peeraddr = s.accept()
    s2.send('These are my actual data')
    s2.close()

Program querying this socket:

MYSOCKETFILE = '/tmp/mysocket'
import socket
import os
s = socket.socket(socket.AF_UNIX)
s.connect(MYSOCKETFILE)
while True:
    d = s.recv(100)
    if not d: break
    print d
s.close()