How do I fix this rabbitmq/celery error?

Question

So I'm running celery on multiple servers with a clustered rabbitmq backend. Recently, anything I do with celery has begun hanging indefinitely, and checking the logs for rabbitmq provides me with this obscure error message:

=ERROR REPORT==== 20-Mar-2013::23:52:25 ===
connection <0.15823.3>, channel 1 - soft error:
{amqp_error,not_found,
        "no binding i-69995906 between exchange 'i-69995906' in vhost 'celery' and queue 'i-69995906' in vhost 'celery'",
        'queue.bind'}

Running rabbitmqctl list_bindings gives me this:

# rabbitmqctl list_bindings -p celery
Listing bindings ...
        exchange    celery  queue   celery  []
celery  exchange    celery  queue   celery  []
...done.

What do I need to do to get rid of the error? I've already restarted Rabbitmq, reinstalled Rabbitmq, and deleted and restored the cluster. I'm guessing that I need to restore the preexisting binding, but I don't know how from rabbitmqctl or celery. If this doesn't work, none of my celery tasks work at all.

Answer 1

Thanks for the question. I ended up in exactly the same place.

I was able to correct this issue by deleting the vhost and recreating it

rabbitmqctl delete_vhost celery
rabbitmqctl add_vhost celery
rabbitmqctl set_permissions -p celery <user> ".*" ".*" ".*"