python: read file continuously, even after it has been logrotated

Question

I have a simple python script, where I read logfile continuosly (same as tail -f)

while True:
    line = f.readline()
    if line:
        print line,
    else:
        time.sleep(0.1)

How can I make sure that I can still read the logfile, after it has been rotated by logrotate?

i.e. I need to do the same what tail -F would do.

I am using python 2.7

Answer 1

As long as you only plan to do this on Unix, the most robust way is probably to check so that the open file still refers to the same i-node as the name, and reopen it when that is no longer the case. You can get the i-number of the file from os.stat and os.fstat, in the st_ino field.

It could look like this:

import os, sys, time

name = "logfile"
current = open(name, "r")
curino = os.fstat(current.fileno()).st_ino
while True:
    while True:
        buf = current.read(1024)
        if buf == "":
            break
        sys.stdout.write(buf)
    try:
        if os.stat(name).st_ino != curino:
            new = open(name, "r")
            current.close()
            current = new
            curino = os.fstat(current.fileno()).st_ino
            continue
    except IOError:
        pass
    time.sleep(1)

I doubt this works on Windows, but since you're speaking in terms of tail, I'm guessing that's not a problem. :)

Answer 2

You can do it by keeping track of where you are in the file and reopening it when you want to read. When the log file rotates, you notice that the file is smaller and since you reopen, you handle any unlinking too.

import time

cur = 0
while True:
    try:
        with open('myfile') as f:
            f.seek(0,2)
            if f.tell() < cur:
                f.seek(0,0)
            else:
                f.seek(cur,0)
            for line in f:
                print line.strip()
            cur = f.tell()
    except IOError, e:
        pass
    time.sleep(1)

This example hides errors like file not found because I'm not sure of logrotate details such as small periods of time where the file is not available.