What are the advantages of using syslog over other logging facilites?

Question

We are using a basic python log server based on BaseHTTPServer to aggregate our python logs on an ubunutu server. This solution has fulfilled our needs... until now. The number of programs dumping to this log server has grown and now the logger is crippling the system.

Now that we are back to the drawing board, we are considering using syslog.

Would it be advantageous to use syslog over other logging facilites.

Thanks for the help

Answer 1

Using syslog might be simple and fast, but it cannot give you full control over how your logs are aggregated.

Your main problem now is using BaseHTTPServer, which was never meant to be used on a production server, or for anything needing high performance.

I see two options:

1. use a better http server with wsgi support, together with mini web framework (we are using gevent+bottle, but http://nichol.as/benchmark-of-python-web-servers is a well written comparison of solutions)
2. use a message queue. this will mean more changes in your code but is a dedicated solution for your problem (and more efficient). (we use rabbitmq but check google or http://www.darkcoding.net/software/choosing-a-message-queue-for-python-on-ubuntu-on-a-vps for comparisons)

Edit: A dedicated solution that supports message queues is logbook. It can also be used as direct replacement of the standard library's logging module.

Answer 2

The advantages of using syslog where available (all modern *nix systems, including Linux, FreeBSD, OS-X etc.) are numerous:

• Performance is better: syslog is compiled C and most importantly it works as a separate process so all your logging operations become non-blocking to the applications, processes, and threads that make them
• You can log from multiple processes/threads concurrently without worrying about locking. All logging is safely serialized for you so you don't lose data
• You get standard sortable time-stamps on all logged lines for free
• You get log rotation for free
• You get severity level support for free (see man syslog)
• You can call logging from any language with a C binding, which is virtually any language
• You can trivially log from shell scripts or command line (via logger)
• You don't need to reinvent the (how to log) wheel

The only disadvantage I can think of is that syslog is non portable (to non *nix systems), but if you're on any modern *nix, any alternative is more complicated and likely less reliable.

The concern of losing packets because syslog is using UDP may be valid, but in practice on a LAN, I've never found it to be an issue.

Answer 3

Not sure why HTTP is needed but this surely adds an overhead. Not to mention that the python implementation might not be up to the task. I can recommend syslog , but beware that using UDP for transport can result in message loss so TCP is highly recommended. syslog-ng and rsyslog can both handle TCP syslog. nxlog can too, and is not even bound to the syslog protocol.