Beanstalkd queue with AWS elastic beanstalk

Question

I wan't to deploy my web app which is running inside docker containers to elastic beanstalk.

When deploying your app to elastic beanstalk you have two environment options you can choose from:

1. Web Server Environment
2. Worker Environment

Logically my webapp uses the first environment type, but I also need to make use of a job queue which is used for long running processes, which would probably run in the second type environment. I don't want to use amazon SQS however because of the vendor lock in I would have when I want to switch to a different host. I want to be running beanstalkd instead, but I can't think off a good solution how to set this up.

I like this answer given by Rohit Banga. This would force me to use SQS though right?

I find it important to have on repository for my code. The workers will use the same code as the web app. Which one repo I think it will be easier to maintain.

I think I'll setup and EC2 instance which will run the Beanstalkd server.

Answer 1

If you are going to run your own queue and do not require (or want) SQS, then just use a webserver.

Web Server and Worker are essentially the same thing. The difference is that a worker tier doesnt have a load balancer. The Worker tier also does all the work of setting up an SQS queue with a deamon that directs queue data to your "web listener".

Since you are not using sqs, you can use a web server. Just know that you will have to make sure your queue works correctly when your environment is scaled. (You dont want to be reading the same item from the queue twice).