How do I properly redirect stdout/stderr from a systemd service on Raspbian?

Question

I set up a service on Raspbian (Jessie) using systemd to make it start after boot. The daemon config looks like this:

[Unit]
After=multi-user.target

[Service]
Type=idle
User=root
ExecStart=/bin/sh -c "exec /home/pi/sources/mydaemon.py >> /home/pi/mydaemon.log 2>&1"

[Install]
WantedBy=multi-user.target

That redirection >> isn't working. I have tried most of the options available to the StandardOutput and StandardError but they never end up printing my script's output to /var/log/daemon.log and journalctl -u mydaemon.service only shows messages about the service being started and stopped.

I'm not doing anything funny with file descriptors from within the script currently. I just want my print() or logging.info() statements to show up somewhere I can read them. Any ideas?

(Just to be clear, the daemon does have to run as root. Could that have something to do with my printing problem?)

Answer 1

I needed to run python with the -u parameter to ensure messages were not buffered.

With these lines, the print lines gets added to the journal immediately:

StandardOutput=journal+console
ExecStart=/home/pengman/scripts/mqtt_monitor/venv/bin/python -u home/pengman/scripts/mqtt_monitor/src/mqtt_monitor.py

(I am running inside a virtualenv, but that shouldn't matter)

Answer 2

Accepted answer from @mark-stosberg is on the money and correct. I wanted to expand his answer a bit and the comments are too short.

The systemd-journald.service man-page mentions this in the STREAM LOGGING section:

The systemd service manager invokes all service processes with standard output and standard error connected to the journal by default. This behaviour may be altered via the StandardOutput=/StandardError= unit file settings, see systemd.exec(5) for details.

So in @eric's original unit file, in order to get the output to /home/pi/mydaemon.log you could do this in the unit file

ExecStart=/usr/bin/python -u /home/pi/sources/mydaemon.py
StandardOutput=file:/home/pi/mydaemon.log
StandardError=inherit

Note the value of StandardError. This is from the systemd.exec(5) manpage, and it means to hook stderr to the same handle as stdout, much like 2>&1 in shell syntax.

Also, if you want the log to update immediately, you must tell Python not to buffer the output (with it's -u switch). That's why I changed the ExecStart= as well. When streaming python output (whether in systemd or from the shell) the output is buffered by default and the file won't be updated until the buffer flushes or the process ends.

---

UPDATE 20190411 - Raspbian's earlier version of systemd does not accept the file:/ target for StandardOutput (thanks @nak for pointing this out), so my answer doesn't really work for the OP question as regards Raspbian (I tested in openSUSE Tumbleweed). This SO question "How to redirect output of systemd service to a file" has details for alternative ways to do it with older systemd.

Answer 3

Normally you just run a service directly (make sure it's executable has a shebang line):

 ExecStart=/home/pi/sources/mydaemon.py

And use the default redirection of StandardOutput= to the systemd journal, so you can read the logs with journalctl -u mydaemon.service.

Systemd nicely controls the file growth and file rotation of the log for you.

It is unrelated that your service runs as root.

If you don't see any log output with the above, also check the entire log for nearby logs that might be attributed to your service:

 journalctl

This is necessary because there's a known issue when some last lines of logging just before a script exists might not be attributed to the script. This would be especially noticeable for a script that just runs a fraction of a second before exiting.