What is celery.utils.log.ProcessAwareLoggerobject doing in logging.Logger.manager.loggerDict

Question

I am inspecting the logging.Logger.manager.loggerDict by doing:

import logging
logging.Logger.manager.loggerDict

and the dict is as follows:

{
  'nose.case': <celery.utils.log.ProcessAwareLoggerobjectat0x112c8dcd0>,
  'apps.friends': <logging.PlaceHolderobjectat0x1147720d0>,
  'oauthlib.oauth2.rfc6749.grant_types.client_credentials': <celery.utils.log.ProcessAwareLoggerobjectat0x115c48710>,
  'apps.adapter.views': <celery.utils.log.ProcessAwareLoggerobjectat0x116a847d0>,
  'apps.accounts.views': <celery.utils.log.ProcessAwareLoggerobjectat0x116976990>,

  }
  There are more but I truncated it

My questions are :

1. How come celery is involved in logging of various other non-celery apps? Is it because logging is done in an async way and somehow logging framework detects presence of celery and uses it?
2. For two of my own files that are logging using logger = logging.getLogger(__name__) , I see one is PlaceHolderObject and other two it is celery.utils.log.ProcessAwareLogger object - although these latter two are called in views and not in celery processes. How did it become this way then

Thanks

Answer 1

Celery itself replaces the (global) logger class, using the logging.setLoggerClass method, with a ProcessAwareLogger class that does a couple of things: avoid trying to log while in a signal handler, and add a process name to logs. This happens as soon as Celery's logging system is set up. You're seeing this class even on your own loggers because of the global nature of setLoggerClass.

As for why, exactly, Celery is designed like that, I think you'd have to ask a developer of Celery, but effectively it allows Celery to ensure that signal handler safety and process name are taken care of even if you use your own loggers in your app.

The python logging docs note:

If you are implementing asynchronous signal handlers using the signal module, you may not be able to use logging from within such handlers. This is because lock implementations in the threading module are not always re-entrant, and so cannot be invoked from such signal handlers.

Celery uses signal so this may be a reason for wanting to globally enforce its logger class.