Parse a log4j log file

Question

We have several applications that use log4j for logging. I need to get a log4j parser working so we can combine multiple log files and run automated analysis on them. I'm not looking to reinvent the wheel, so can someone point me to a decent pre-existing parser? I do have the log4j conversion pattern if that helps.

If not, I'll have to roll our own.

Answer 1

I didn't realize that Log4J ships with an XML appender.

Solution was: specify an XML appender in the logging configuration file, include that output XML file as an entity into a well formed XML file, then parse the XML using your favorite technique.

The other methods had the following limitations:

• Apache Chainsaw - not automated enough
• jdbc - poor performance in a high performance distributed app

Answer 2

You can use OtrosLogViewer with batch processing. You have to:

1. Define you log format, you can use Log4j pattern layout parser or Log4j XmlLayout
2. Create java class that implements LogDataParsedListener. Method public void logDataParsed(LogData data, BatchProcessingContext context) will be called on every parsed log event.
3. Create jar
4. Run OtrosLogViewer with specifying your log processing jar, LogDataParsedListener implementation and log files.