Can't get celery to do INFO logging the same way Django does it

Question

I have my logging configured like so:

LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'handlers': {
        'mail_admins': {
            'level': 'INFO',
            'class': 'common.handlers.SuperAdminEmailHandler'
            },
        'console': {
            'level': 'INFO',
            'class': 'logging.StreamHandler'
            }
        },
    'loggers': {
        '': {
            'handlers': ['console'],
            'level': 'INFO',
            'propagate': True,
        },
        'django.request': {
            'handlers': ['console'],
            'level': 'ERROR',
            'propagate': True,
            },
        },
    }

This is with CELERYD_HIJACK_ROOT_LOGGER = False. I also have this task:

@periodic_task(run_every=crontab())
def test_logging():
    import logging
    print('running test_logging')
    logging.info("Here's an info message", extra = {
        'tell_admins': True, 'email_content': 'Additional content'
    })
    logging.error("Here's an error message")

I have a test URL that calls test_logging directly. When I hit that URL I see this output:

running test_logging
Here's an info message
Here's an error message
[05/Jul/2013 11:07:27] "GET /test/ HTTP/1.1" 200 7

Exactly what I expect. When that same function runs through celery though, I get:

Scheduler: Sending due task scheduler.tasks.test_logging (scheduler.tasks.test_logging)
running test_logging
Here's an error message

Where'd the info message go?! I tried adding another celery-specific logger:

    'celery': {
        'handlers': ['console'],
        'level': 'INFO',
        'propagate': True,
        },

Now I see this from celery:

running test_logging
running test_logging
Here's an error message
Task scheduler.tasks.test_logging[271b8a4a-0c04-4391-81c5-5b009d70b08d] succeeded in 0.00929498672485s: None

Still no info message. Any thoughts on why and how to get it back?

Answer 1

For some reason, even if Celery is using then handlers you specify in Django's logging settings, it still overrides the levels of the loggers. I'm now using this function to get a logger. With the logger set to the appropriate level, things seem to be working as expected.

import logging

from celery.utils.log import get_task_logger

def get_celery_logger(name):
    logger = get_task_logger(name)
    logger.level = logging.INFO
    return logger