Celery scheduled tasks problems with Timezone

Question

I'm using celery in a server where server time is now BST, and suddenly my scheduled tasks are executing one hour before! Previously, server time was Europe/London which was GMT but now due to day light saving it has become BST (GMT + 1)

I've configured celery to use the timezone like:

CELERY_TIMEZONE = 'Europe/London'

Then when calling tasks, I've also localized value for the eta parameter to 'Europe/London' like this:

from datetime import datetime
from pytz import timezone

locale_to_use = timezone('Europe/London')
current_time = locale_to_use.localize(datetime.now())

And used this current_time as value of eta parameter when calling task.

Now is there any mistake I'm making like localizing the eta parameter value? My server is in BST.

No problems were hapenning with this configuration before the day light saving timezone was in effect!

Edit:

To make things clear, I'm posting my code samples here:

@app.task(ignore_result=True)
def eta_test():

    logger.info('Executing eta_test on {0}'.format(datetime.now()))


def run_eta_test(hour, minute):

    now_time = datetime.now()
    target_time = now_time.replace(hour=hour, minute=minute, second=0)

    from settings import options
    from pytz import timezone

    local_zone = timezone('Europe/London')

    target_time = local_zone.localize(target_time)

    eta_test.apply_async(eta=target_time)

Then I call run_eta_test from a python console in the server, like this:

test_tasks.run_eta_test(17, 17)

That is, to execute the task on 17:17:00 on the same day. The server time was 17:15:45 and instead of scheduling it 2 seconds after, it executed the task immediately:

[2014-04-04 17:15:45,341: INFO/MainProcess] Received task: scheduling.test_tasks.eta_test[c28448d6-3a51-42f7-9df2-cb93385ff7c6] eta:[2014-04-04 17:17:00.095001+01:00]
[2014-04-04 17:15:46,820: INFO/Worker-3] Executing eta_test on 2014-04-04 17:15:46.820316
[2014-04-04 17:15:46,820: INFO/MainProcess] Task scheduling.test_tasks.eta_test[c28448d6-3a51-42f7-9df2-cb93385ff7c6] succeeded in 0.0008487419690936804s: None

Then I called the task again to be executed 1 hour and few seconds later by calling like:

test_tasks.run_eta_test(18, 17)

And instead of scheduling it to one hour and few seconds later, the task executed only few seconds later, that is one hour before:

[2014-04-04 17:16:27,703: INFO/MainProcess] Received task: scheduling.test_tasks.eta_test[f1a54d08-c12d-457f-bee8-04ca35b32242] eta:[2014-04-04 18:17:00.700327+01:00]
[2014-04-04 17:17:01,846: INFO/Worker-2] Executing eta_test on 2014-04-04 17:17:01.846561
[2014-04-04 17:17:01,847: INFO/MainProcess] Task scheduling.test_tasks.eta_test[f1a54d08-c12d-457f-bee8-04ca35b32242] succeeded in 0.0012819559779018164s: None

My server date is configured to BST like:

Fri Apr  4 17:29:10 BST 2014

Now, is the timezone in the server is being an issue?

Edited Again:

I was not able to solve the problem with the help of the answer and comments. So what I did is, used: CELERY_ENABLE_UTC = False and did not use any value for CELERY_ENABLE_UTC at all. Then, I used my server time without any localization. Celery seemed to schedule tasks correctly at my server time.

Answer 1

This was a bug in Celery 4 that I helped contribute the fix for in Celery 4.2 -- it is true that the work around was to set the project's timezone for celery to be UTC, however as of Celery 4.2 you can use whatever timezone you want again and celerybeat scheduling will work properly. More details in the original bug report/PR: https://github.com/celery/celery/pull/4324