Menu
I'm using Celery with an AMQP broker to call tasks, but the response needs to be passed back with a different queue architecture than Celery uses, so I want to pass the messages back using Kombu only. I've been able to do this, but I'm creating a new connection every time. Does Celery use a broker connection pool, and if so, how do you access it?
961
The broker is the third-person facilitator between a buyer and a seller. Celery requires a solution to send and receive messages; usually, this comes in the form of a separate service called a message broker. In celery, the broker is Redis, RabbitMQ, etc who conveying the message between a client and celery.
celery -A merlin worker --concurrency 2. The --prefetch-multiplier argument sets how many tasks are requested from the task server per worker thread. If --concurrency is 2 and --prefetch-multiplier is 3, then 6 tasks will be requested from the task server by the worker threads.
What is a Celery worker? Celery is a task management system that you can use to distribute tasks across different machines or threads. It allows you to have a task queue and can schedule and process tasks in real-time. This task queue is monitored by workers which constantly look for new work to perform.
It took a lot of searching because Celery's documentation is... wonderful... but I found the answer.
Celery does use a broker connection pool for calling subtasks. The celery application has a pool attribute that you can access through <your_app>.pool or celery.current_app.pool. You can then grab a connection from the pool using pool.acquire().
65
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
