How to tail a file with gevent

Question

I have a problem constructing gevent tail function. In general, the code works when I comment gevent.sleep in loop, but then CPU utilization is 100%. When I leave gevent.sleep program works but nothing is happening. Gevent version is 1.0b1.

import os
import gevent

def follow(filename):
    fd = os.open(filename, os.O_RDONLY|os.O_NONBLOCK)
    os.lseek(fd, 0, os.SEEK_END)
    hub = gevent.get_hub()
    watcher = hub.loop.io(fd, 1)
    while True:
        hub.wait(watcher)
        lines = os.read(fd, 4096).splitlines()
        if not lines:
            #gevent.sleep(.1)
            continue
        else:
            for line in lines:
                print "%s:%s" % (filename, line)

    os.close(fd)

if __name__ == '__main__':
    job1 = gevent.spawn(follow, '/var/log/syslog')
    job2 = gevent.spawn(follow, '/var/log/messages')

    gevent.joinall([job1, job2])

Answer 1

Starting with gevent 1.0b2 you can use stat watchers to get notifications on file changes.

See the code and the libev documentation for stat watchers.

Answer 2

Well, that's code doesn't 'tail' the file, it's just prints the whole file, BUT it's show's how 'loop.stat' works. It wait's for file to change - or be just touched, and then print's content. When it's wait - it take's almost no resources!

import gevent,os

def follow(filename):
    hub = gevent.get_hub()
    watcher = hub.loop.stat(filename)
    while True:
        hub.wait(watcher)
        with open(filename) as f:
            print f.read()

if __name__ == '__main__':
    jobs=[gevent.spawn(follow,'/var/log/syslog')]
    jobs+=[gevent.spawn(follow,'/var/log/messages')]
    gevent.joinall(jobs)