I want to upload images to S3 server, but before uploading I want to generate thumbnails of 3 different sizes, and I want it to be done out of request/response cycle hence I am using celery. I have read the docs, here is what I have understood. Please correct me if I am wrong.
Now my understanding is - if multiple users upload image at the same time, celery will queue the resizing, and the resizing will actually happen at the ironMQ server, since it offers a cool addon on heroku.
Now the doubts:
But what after the image is resized, will ironMQ push it to the S3 server, or will it notify once the process is completed.. i am not clear about it.
What is the difference between celery and kombu/carrot, could you explain vividly.
PyAMQP is a Python library that lets Python clients communicate with any implementation of AMQP, including RabbitMQ.
`Kombu` is a messaging library for Python. The aim of `Kombu` is to make messaging in Python as easy as possible by. providing an idiomatic high-level interface for the AMQ protocol, and also. provide proven and tested solutions to common messaging problems.
IronMQ does not process your tasks for you; it simply serves as the backend for Celery to keep track of what jobs need to be performed.
So, here's what happens. Assume you have two servers, your web server and your Celery server. Your web server is responsible for handling requests, your Celery server creates the thumbnails and uploads them to S3. Here's what a typical request looks like:
I wrote an example for using IronMQ on Heroku. You can see it here: http://iron-celery-demo.herokuapp.com. You can see the source for the example on Github and read the tutorial, which explains pretty thoroughly and step-by-step how to deploy Celery on Heroku.
To clear up the AMQP stuff:
One of the biggest differences between IronMQ and RabbitMQ/AMQP is that IronMQ is hosted and managed, so you don't have to host the server yourself and worry about uptime. The spec offers a bunch more in terms of differentiation, and there are underlying differences, but Celery abstracts most of those away. Because you're using Celery, the only difference you're liable to notice is that IronMQ is hosted, so you don't have to stand up and manage your own server.
Full disclosure: I am employed by Iron.io, the company behind IronMQ.
"One of the biggest differences between IronMQ and RabbitMQ/AMQP is that IronMQ is hosted and managed, so you don't have to host the server yourself and worry about uptime."
Currently there are at least two hosted managed RabbitMQ-as-a-service options: Bigwig and CloudAMQP. Celery should work well with both.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With