Google App Engine Task Queue

Question

I want to run 50 tasks. All these tasks execute the same piece of code. Only difference will be the data. Which will be completed faster ?

a. Queuing up 50 tasks in a queue

b. Queuing up 5 tasks each in 10 different queue

Is there any ideal number of tasks that can be queued up in 1 queue before using another queue ?

Answer 1

The rate at which tasks are executed depends on two factors: the number of instances your app is running on, and the execution rate of the queue the tasks are on.

The maximum task queue execution rate is now 100 per queue per second, so that's not likely to be a limiting factor - so there's no harm in adding them to the same queue. In any case, sharding between queues for more execution rate is at best a hack. Queues are designed for functional separation, not as a performance measure.