Log4J RollingFileAppender Unable to Roll File

Question

I am writing a small xml transformation layer in Java. I receive xml via web service, modify it, and then send the modified xml to another system. I then wait for a response and return the response to the original caller.

System A -> Me -> System B -> Me -> System A

I want to log the request I receive, the request I send, the response I receive, and the request I send. Basically I want to log the xml where each arrow is in my diagram.

My problem is with the RollingFileAppender. I try to roll at 10MB, sometimes it does and sometimes it doesn't roll. If it rolls a couple times, and then stops, it will continue to rename the rolled files from 3 to 4 and 4 to 5 and so on.

My best guess is that when the 10MB mark is crossed, there are multiple threads writing to the log file so the file cannot me renamed. I am hoping that Log4J has an easy solution for this, but if necessary, I am open to switching to a new logging framework. Thank you in advance for any help.

EDIT Here is my properties file.

log4j.rootLogger=DEBUG, fileOut

log4j.appender.fileOut=org.apache.log4j.RollingFileAppender
log4j.appender.fileOut.File=/logs/log.log
log4j.appender.fileOut.layout=org.apache.log4j.PatternLayout
log4j.appender.fileOut.layout.ConversionPattern=%d %-5p %c - %m%n
log4j.appender.fileOut.MaxFileSize=10MB
log4j.appender.fileOut.MaxBackupIndex=10
log4j.appender.fileOut.append=true

EDIT 2 This is essentially a bump, as this post has a low number of views. I feel like this cannot be a unique problem. Any help is much appreciated. Thanks!

Answer 1

Log4J initializes itself at the classloader level. Within a certain classloader and its ancestors, Log4J can only be initialized once and the same Log4J configuration applies to all Log4J calls within the classloader.

As long as all of your logging calls are performed within the same Log4J configuration "realm", Log4J knows how to synchronize access to the physical file pointed at by the rolling appender configuration; when the time comes to roll, rolling is performed with no problem.

Things become problematic once you have two (or more) Log4J "configuration realms" using the same physical file for the rolling appender configuration. That could be:

1. Two different web applications on the same physical JVM
2. Two different web applications on two distinct JVMs
3. Same web application horizontally clustered on two distinct JVMs

(etc)

Log4J simply has no way of knowing who else, other than itself within the same Log4J configuration realm, uses that file. So, what ends up happening is that Log4J on System A attempts to roll the file (because it thinks that no other processes are accessing that file), and fails because someone on System B is using the file at the same time.

This is a known limitation of using file appenders, and you can't really blame Log4J for this: Log4J simply doesn't have the means of monitoring who else, other than Log4J in the same "configuration realm", is using that file.

For such usage scenario, you can use the Log4J socket appender.

If your scenario doesn't involve multiple Log4J "configuration realms", then try adding -Dlog4j.debug=true to the JVM parameters and see what exactly is going on during the file rolling operation.

Answer 2

For others that arrive here, check you are using RollingFileAppender NOT FileAppender!

Cut and paste errors are too easy, for me at least.