Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How do Google App Engine Task Queues work?

I'm confused about Task execution using queues. I've read the documentation and I thought I understood bucket_size and rate, but when I send 20 Tasks to a queue set to 5/h, size 5, all 20 Tasks execute one after the other as quickly as possible, finishing in less than 1 minute.

deferred.defer(spam.cookEggs, 
               egg_keys, 
               _queue="tortoise")  

- name: tortoise
  rate: 5/h  
  bucket_size: 5  

What I want is whether I create 10 or 100 Tasks, I only want 5 of them to run per hour. So it would take 20 Tasks approximately 4 hours to complete. I want their execution spread out.

UPDATE

The problem was I assumed that when running locally, that Task execution rate rules were followed, but that is not the case. You cannot test execution rates locally. When I deployed to production, the rate and bucket size I had set executed as I expected.

like image 821
Will Curran Avatar asked Feb 09 '11 21:02

Will Curran


1 Answers

Execution rates are not honored by the app_devserver. This issue should not occur in production.

[Answer discovered by Nick Johnson and/or question author; posting here as community wiki so we have something that can get marked accepted]

like image 106
Charles Duffy Avatar answered Nov 12 '22 13:11

Charles Duffy