Production settings file for log4j?

Question

Here is my current log4j settings file. Are these settings ideal for production use or is there something I should remove/tweak or change? I ask because I was getting all my threads being hung due to log4j blocking. I checked my open file descriptors I was only using 113.

# ***** Set root logger level to WARN and its two appenders to stdout and R.
log4j.rootLogger=warn, stdout, R

# ***** stdout is set to be a ConsoleAppender.
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
# ***** stdout uses PatternLayout.
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
# ***** Pattern to output the caller's file name and line number.
log4j.appender.stdout.layout.ConversionPattern=%5p [%t] (%F:%L) - %m%n

# ***** R is set to be a RollingFileAppender.
log4j.appender.R=org.apache.log4j.RollingFileAppender
log4j.appender.R.File=logs/myapp.log
# ***** Max file size is set to 100KB
log4j.appender.R.MaxFileSize=102400KB
# ***** Keep one backup file
log4j.appender.R.MaxBackupIndex=5
# ***** R uses PatternLayout.
log4j.appender.R.layout=org.apache.log4j.PatternLayout
log4j.appender.R.layout.ConversionPattern=%p %t %d %c - %m%n


#set httpclient debug levels
log4j.logger.org.apache.component=ERROR,stdout 
log4j.logger.httpclient.wire=ERROR,stdout 
log4j.logger.org.apache.commons.httpclient=ERROR,stdout 
log4j.logger.org.apache.http.client.protocol=ERROR,stdout

UPDATE*** Adding thread dump sample from all my threads (100)

"pool-1-thread-5" - Thread t@25
   java.lang.Thread.State: BLOCKED on org.apache.log4j.spi.RootLogger@1d45a585 owned by: pool-1-thread-35
    at org.apache.log4j.Category.callAppenders(Category.java:201)
    at org.apache.log4j.Category.forcedLog(Category.java:388)
    at org.apache.log4j.Category.error(Category.java:302)

Answer 1

Log4j 1.2 is vulnerable to deadlocks when toString() produces nested logging.

See old aged still unresolved issues like Log4J can create deadlock conditions (concurrent package donation) and Deadlock with RollingFileAppender.

It also has performance-killing lock synchronization issues under heavy concurrent load. Like Category callAppenders synchronization causes java.lang.Thread.State: BLOCKED and Move org.apache.log4j.Category to reentrant read/write locks.

Even AsyncAppender is not free of excessive locks: AsyncAppender.doAppend() does not need to be synchronized and Deadlock in 1.2.15 caused by AsyncAppender and ThrowableInformation classes. Also beware of AsyncAppender overflow.

One caveat is to always constrain root category level to at least INFO or higher. This would prevent logging calls from acquiring unnecessary locks mentioned in the above issues. Just limiting appender threshold is not sufficient as it is taken into account later. See explanation with publish/subscribe analogy:

To answer your question about how threshold interacts with category, basically think of it is as a publish/subscribe. The category sets what is published by the logger, the threshold sets the subscription level of the appender.

Nested categories of interest can be individually given lower priorities if needed.

Answer 2

%F:%L has serious performance impact. Although I don't see how they'd cause locking, I'd consider omitting them for production.