Centralized Logging for many Java Apps: Syslog vs JMS vs Http vs Local file

Question

I want all my applications log to be centralized (ideally in near real-time). We will use a Log4 Appender. Which one should I use:

1. Send log event in a JMS Queue
2. Syslog / syslog-ng
3. Write to a localfile and use rsync (every 3second) to replicate the log
4. Do a POST to a centralized REST Http Service

Which one are you using?

Answer 1

Depends on your requirements. Logging to a queue (as in JMS) gives you the most flexibility of your options as your log operation can return as soon as the log message has been written to the queue. You are then free to have another process take those log messages off the queue and write them to your preferred logging store (database, file system, ...).

The (slight) downside is your log messages will lag slightly behind the system that is logging, but this is almost always the case, even with file-based logging.

Answer 2

This may be overkill but have you looked at Splunk? For Java apps, I've used either Log4J or SLF4J to log to files, then Splunk can aggregate the logs and make them searchable.