Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Sustainable Solution To Configuring Rails, Sidekiq, Redis All On AWS Elastic Beanstalk

AWS Elastic Beanstalk rails app that needs a sidekiq worker processes running alongside Puma/Passenger. Getting the sidekiq process to run has resulted in hours failed attempts. Also, getting the rails app and sidekiq to talk to my AWS ElastiCache cluster apparently needs some security rule changes.

Background

We started out with an extremely simple Rails app that was easily deployed to AWS Elastic Beanstalk. Since those early times we've evolved the app to now use the worker framework Sidekiq. Sidekiq in turn likes to use Redis to pull its jobs. Anyway, getting all these puzzle pieces assembled in the AWS world is a little challenging.

like image 889
Dan Avatar asked Jun 25 '15 16:06

Dan


People also ask

Does Sidekiq need Redis?

Sidekiq uses simple and efficient background processing. Sidekiq is supported by Redis as a job management tool to process thousands of jobs in a second. Follow the steps to add Sidekiq and Redis to your existing application.

What does Sidekiq store in Redis?

Sidekiq is one of the more widely used background job frameworks that you can implement in a Rails application. It is backed by Redis, an in-memory key-value store known for its flexibility and performance. Sidekiq uses Redis as a job management store to process thousands of jobs per second.


1 Answers

Solutions From The Web...with some sustainability problems

The AWS ecosystem goes through updates and upgrades, many aren't documented with clarity. For example environment settings change regularly; a script you have written may break in subsequent versions.

I used the following smattering of solutions to try to solve this:

  • http://blog.noizeramp.com/2013/04/21/using-sidekiq-with-elastic-beanstalk/ (please note that the comments in this blog post contains a number of helpful gists). Many thanks to the contributor and commenters in this post.
  • http://qiita.com/sawanoboly/items/d28a05d3445901cf1b25 (starting sidekiq with upstart/initctl seems like the simplest and most sustainable approach). This page is in japanese, but the sidekiq startup code makes complete sense. Thanks!
  • Use AWS's ElastiCache for Redis. Make sure to configure your security groups accordingly: this AWS document was helpful...
like image 52
Dan Avatar answered Oct 21 '22 14:10

Dan