multithreaded logging for high performance application [duplicate]

Question

I have an application (server application) that needs an extensive amount of logging implemented and shouldn't be too affected performance wise by enabling logging.

The application has a thread pool of worker threads performing the work. Originally I was going to just log on these thread pool threads, but then I'd need to lock practically the whole thread and so there goes my 'multithreaded' app.

I have been looking at better ways to log from multiple threads and I've found using a Queue or ring buffer could be an idea.

Can anybody suggest (perhaps from experience) any good ways to implement effective logging (to a file mostly) for a multithreaded application that also should stay somewhat performant?

I would like to use the Boost Logging Library.

Answer 1

Pantheios is the fastest logging library for C++ out there, as far as I know. I recommend using it instead of Boost Logging. With Pantheios you simply log to the file, and you don't care from which thread. You can put the thread name in the logline prefix if you want and it does everything for you.

Answer 2

Personally I would look into Pantheios, gave it a glance and it seems interesting, going to include that in a future project of mine.

If you really want to use boost logging I would use a synchronized queue that handles all the locking internally so your workers doesn't have to worry about that.

pseudocode:

 class SynchedQueue {
      void write(text) { lock() logfile.write(text) unlock() }

Or if you really want to make it fast, use an internal queue and transfer from the public queue in batches of X lines at a time. The public queue doesn't have to wait for file i/o that might take a relativily long time and the private queue only gets lines seldomly. Although it will probably still be slower than Pantheios.