Reading systemd journal from Python script

Question

I am trying to emulate this shell command in Python using the systemd libraries http://www.freedesktop.org/software/systemd/python-systemd/journal.html

I am actually trying to emulate this command but within Python.

journalctl --since=-5m --no-pager

I have seen others do this in Python by calling the journal executable but that is a pretty bad way of doing it.

I wrote this simple script based on the documentation linked above

import select
from systemd import journal

j = journal.Reader()
j.log_level(journal.LOG_INFO)
# j.add_match(_SYSTEMD_UNIT="systemd-udevd.service")

j.seek_tail()
j.get_next()

while j.get_next():
    for entry in j:
        if entry['MESSAGE'] != "":
            print(str(entry['__REALTIME_TIMESTAMP'] )+ ' ' + entry['MESSAGE'])

There are a few issues here

1. the log seems to start from about 5 days ago which means the seek_tail did not appear to work.
2. I am getting a lot of junk in here is there a specific filter I should use to match the data I get from journalctl command given at the beginning of the question?

Ideally longer term I want to just be following this journal based on a set of filters/matches to emulate the command 'journalctl -f' but I just need to resolve this issue first. I want to end up with something like this but it doesnt work either.

import select
from systemd import journal

j = journal.Reader()
j.log_level(journal.LOG_INFO)

# j.add_match(_SYSTEMD_UNIT="systemd-udevd.service")
j.seek_tail()

p = select.poll()
p.register(j, j.get_events())
while p.poll():

    while j.get_next():
        for entry in j:
            if entry['MESSAGE'] != "":
                print(str(entry['__REALTIME_TIMESTAMP'] )+ ' ' + entry['MESSAGE'])

Answer 1

I am also working on similar python module.

According to the following links, we have to call sd_journal_previous (In python systemd module, that is journal.Reader().get_previous()).

http://www.freedesktop.org/software/systemd/man/sd_journal_seek_tail.html

https://bugs.freedesktop.org/show_bug.cgi?id=64614

In addition, your example code will consume 80 - 100% CPU load, because the state of reader remains "readable" even after get an entry, which results in too many poll().

According to the following link, it seems we have to call sd_journal_process (In python systemd module, that is journal.Reader().process()) after each poll(), in order to reset the readable state of the file descriptor.

http://www.freedesktop.org/software/systemd/man/sd_journal_get_events.html

In conclusion, your example code would be:

import select
from systemd import journal

j = journal.Reader()
j.log_level(journal.LOG_INFO)

# j.add_match(_SYSTEMD_UNIT="systemd-udevd.service")
j.seek_tail()
j.get_previous()
# j.get_next() # it seems this is not necessary.

p = select.poll()
p.register(j, j.get_events())

while p.poll():
    if j.process() != journal.APPEND:
        continue

    # Your example code has too many get_next() (i.e, "while j.get_next()" and "for event in j") which cause skipping entry.
    # Since each iteration of a journal.Reader() object is equal to "get_next()", just do simple iteration.
    for entry in j:
        if entry['MESSAGE'] != "":
            print(str(entry['__REALTIME_TIMESTAMP'] )+ ' ' + entry['MESSAGE'])