Menu
I'm using Celery 2.4.6 and django-celery 2.4.2.
When I configure Celery to use Amazon SQS per the resolution on this question: Celery with Amazon SQS
I don't see anything in the celerycam table in the Django admin. If I switch back to RabbitMQ, the tasks start showing up again.
I have a lot (now 40+) queues in SQS named something like this: "celeryev-92e068c4-9390-4c97-bc1d-13fd6e309e19", which look like they might be related (some of the older ones even have an event in them), but nothing's showing up in the database and I see no errors in the celerycam log.
Any suggestions on what the issue might be or how to debug this further would be much appreciated.
438
In most cases, you can set the ReceiveMessage wait time to 20 seconds. If 20 seconds is too long for your application, set a shorter ReceiveMessage wait time (1 second minimum).
An SQS Queue can also be configured with a Message Retention Period in seconds. This value specifies how long a message can stay on a queue before it is automatically deleted, regardless of its processing status. The retention period can be set between 60 seconds and 14 days, with the default at 4 days.
Q: How long can I keep my messages in Amazon SQS message queues? Longer message retention provides greater flexibility to allow for longer intervals between message production and consumption. You can configure the Amazon SQS message retention period to a value from 1 minute to 14 days. The default is 4 days.
The first-in-first-out (FIFO) queue is the type of AWS SQS queue that guarantees order and provides exactly once delivery of messages.
SQS is a limited implementation of an AMQP bus. As I understand, it doesn't support PUB/SUB broadcasting like say rabbit-MQ does, which is necessary for events to work properly. SNS was put in place to support broadcasting, but its a separate system.
Some libraries/packages out there are using SimpleDB as a messaging model store as a hack on top of SQS to emulate proper AMQP behavior, but apparently celery does not have a full hack in place yet.
85
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
