Log4j logging threshold

Question

Current logging threshold that is all about log level(TRACE, DEBUG, INFO, WARN, ERROR and FATAL) is not enough for me. I have a gigabytes of log written from third party libraries into ERROR category in emergency case. I do not want to turn of this logging cause I want to see this problem logs. Most of logs are continuously repeating stack traces. So I want a kind of appender which will

• 1)Skip logs if threshhold(kb/sec) is reaсhed (I mean when we are writing to much logs - we might skip some) or
• 2)Skip stacktraces printing if more then one(n) stacktrace was printed in period of time

Please suggest

Answer 1

Log4j do not have such an appender out of box. But you may do following:

Add appenders for third party libraries which uses gigabytes of data ("I have a gigabytes of log written from third party libraries into ERROR category"). And configure your appenders such that they do not use such a large amount of storage.

1. RollingFileAppender Configure this third party library appender as rolling file. After some time they will use oldest file and you can keep only latest logs.

2. JDBCAppender This appender accepts usage of stored procedures. Write a stored procedure so that it will do what you want.

For your first requirement add a calculated column or calculate this value in stored procedure and add to to your table columns. Then control this value to decide if you need to insert this log line to database.

For your second requirement You can take hash value (MD5,SHA etc) of exception message. If this hash value exists in database table you may ignore to insert it. Or you may count how many of it exists and decide how to insert to database accordingly.

You do not need to use enterprise database for these purposes, you may use apache derby for example, then all of your logs stay in application server. I think this one,JDBCAppender, meets your requirements more closely.