I'm confused about Task execution using queues. I've read the documentation and I thought I understood bucket_size and rate, but when I send 20 Tasks to a queue set to 5/h, size 5, all 20 Tasks execute one after the other as quickly as possible, finishing in less than 1 minute.
deferred.defer(spam.cookEggs,
egg_keys,
_queue="tortoise")
- name: tortoise
rate: 5/h
bucket_size: 5
What I want is whether I create 10 or 100 Tasks, I only want 5 of them to run per hour. So it would take 20 Tasks approximately 4 hours to complete. I want their execution spread out.
The problem was I assumed that when running locally, that Task execution rate rules were followed, but that is not the case. You cannot test execution rates locally. When I deployed to production, the rate and bucket size I had set executed as I expected.
Execution rates are not honored by the app_devserver. This issue should not occur in production.
[Answer discovered by Nick Johnson and/or question author; posting here as community wiki so we have something that can get marked accepted]
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With