Unknown queue names show on Rabbitmq mgmt. when using Celery

Question

I only created the last 2 queue names that show in Rabbitmq management Webui in the below table:

The rest of the table has hash-like queues, which I don't know:

1- Who created them? (I know it is celery, but which process, task,etc.)

2- Why they are created, and what they are created for?.

I can notice that when the number of pushed messages increase, the number of those hash-like messages increase.

Answer 1

When using celery, Rabbitmq is used as a default result backend, and also to store errors of failing tasks(that raised exceptions).

Every new task creates a new queue on the server, with thousands of tasks the broker may be overloaded with queues and this will affect performance in negative ways.

Each queue in Rabbit will be a separate Erlang process, so if you’re planning to keep many results simultaneously you may have to increase the Erlang process limit, and the maximum number of file descriptors your OS allows.

Old results will not be cleaned automatically, so we have to tell rabbit to do so.

The below conf. line dictates the time to live of the temp queues. The default is 1 day

CELERY_AMQP_TASK_RESULT_EXPIRES = Number of seconds

OR, We can change the backend store totally, and not make it in Rabbit.

CELERY_BACKEND = "amqp"

We may also ignore it:

CELERY_IGNORE_RESULT = True.

Also, when ignoring the result, we can also keep the errors stored for later usage, which means one more queue for the failing tasks.

CELERY_STORE_ERRORS_EVEN_IF_IGNORED = True.

I will not mark this question as answered, waiting for a better answer.

Rererences:

• This SO link
• Celery documentation
• Rabbitmq documentation