There are huge numbers of threads running in parallel continuously (let's assume this continuous part)). All the threads want to log some application data, basically a set of values.
Several threads like this and this suggest log4net and log4j but I want to know the actual process? Also how multiple threads write to same log file? Is file level lock required for each thread? How does all this work?
Any pointer towards understanding all the details would be appreciated.
Thanks.
A library like log4j will be able to be configured for your needs.
Splitting into too many files will make it difficult to debug some issues, but having one monolithic file leaves a soup of mixed processes. I would have a file for each atomic process, that is, a mail manager might use its own log file. Extra debug information for jdbc might have its own log file, but errors and major events would still be reported in the main application log.
Major logging libraries support log splitting and rotation. For a well used web application, I prefer to have a log file made for each day, and also split over a certain size. You can build a cron to zip older logs and depending on the application, you may want to back them up for a few months or indefinitely.
As far as debugging usefulness, you can grep for certain strings such as "Exception" to report on. If you are looking for statistics, you should make a log for that specific purpose in addition to your process log.
Logs can be synchronous or asynchronous, and the latter is usually best for performance. In general, a queue of messages is built and then written by a separate thread. So multiple threads can write to that one queue or buffer in memory and one thread will lock and write the file. Its pretty much in the background and you don't have to think about it unless you are writing a huge amount of data.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With