High level instructions for migrating application from Heroku to AWS?

Question

I am a novice system administrator, looking for some general guidance on how to migrate an entire Rails application from Heroku to AWS (elastic beanstalk). I use very little amount of heroku services (other than sendgrid). My biggest obstacle is integration of the application itself and migrating the database (will be from heroku postgresql to AWS RDS postgresql).

Would appreciate any insights from those who have gone through the same migration. Thank you!!!

Answer 1

I’ve migrated a couple of clients from Heroku to Elastic Beanstalk, so I’ve put together these high-level instructions on how to set up your Rails application to work on Elastic Beanstalk:

Web processes

Within Elastic Beanstalk, you have multiple ways in which you can run your Rails application.

You can use Elastic Beanstalk’s Ruby platform to deploy your Rails application directly on a preconfigured EC2 instance. It will require you to learn how Amazon chose to set up this platform and how you’re supposed to integrate your application with it.

If you have a little Docker experience, a better approach in my opinion would be to use Elastic Beanstalk’s Docker platform and deploy your Rails application as a Docker container.

You can use Heroku’s cedar Docker image as a base for your Rails container, or you can create your own Rails image by following this tutorial. Either way, assuming you have a basic experience with Docker, you’ll quickly end up with a deployable artifact which will be less dependent on the environment it’s running on.

Just pay attention to the fact that at the moment Elastic Beanstalk has two versions of its Docker platform. The first version allows you to run one Docker container per EC2 instance. The second version uses AWS ECS (Elastic Container Service) to schedule multiple Docker containers on the same EC2 machine. I recommend that you use the second version even if you think you won’t need the multicontainer functionality.

Background jobs

If you’re using asynchronous workers as part of your Rails application, there are a couple of ways you can make them run on Elastic Beanstalk.

You can use the Docker’s multicontainer platform to run the worker process along with your Rails web process on the same EC2 machines. You can utilize a similar approach for the regular, non Docker Rails platform. Use “.ebextensions” to configure the worker daemon on all the EC2 instances that run your web processes.

The main disadvantage of this approach is that you won’t be able to scale your worker tier capacity separately from the web tier. And depending on the nature of your applications, the worker and the web process may compete for resources which also can be undesirable.

Another approach is to use Elastic Beanstalk’s worker environment. You still will be able to use the same Docker/Rails platforms, but Elastic Beanstalk won’t attach a load balancer to your worker applications. Instead, it will create an SQS queue and run an agent on each EC2 machine running your application which will post the content of an SQS message to your application. There is no official support for other queues than SQS, however you can always make your application ignore the local SQS agent and just listen to any other queuing system you would like to use.

Database migration

As Richard mentioned in his answer, the simplest way to migrate the Postgresql database from Heroku to RDS (Amazon’s managed database service) will require some sort of a downtime of your application while you export and import the database data from one platform to another. Fortunately, in most of the migrations I’ve done, this wasn’t a huge problem, it was always possible to find the time when a small maintenance window could be scheduled, usually during the times of day when the site has less active users.

Another important topic to cover is how to migrate Heroku addons you’re using in your applications to Elastic Beanstalk. You mentioned that you use very little of third-party services, therefore I won’t cover how to migrate these services along your application to AWS.

If you want to learn more about Heroku addons migration, I’ve published an article which goes into details on how to replicate Heroku addons on AWS.

Hope that helps.

Answer 2

I've not migrated to AWS, but I've done Heroku > Rackspace, and I'll tell you it's relatively simple if you have the dependencies sorted properly.

There are two real issues:

1. The app itself
2. The database

The app can be handled with GIT. The database will have to be a manual transfer.

--

App Transfer

App transfer is the simplest part - no doubt you have invoked git already due to your using Heroku. This means that you just have to get git set up on your external server.

As a note - if you have any Heroku-based asset dependencies, you need to get those saved locally & migrated. CDN and other repositories - that are Heroku dependent - should be backed up and transferred to the new host. This might not be necessary, but is something to bear in mind.

For the transfer itself, there is a very good tutorial on GoRails here.

In short, you have to set up your server to accept incoming requests with a web server application (nginx / apache). This will then allow you to set up a separate git repo into which you can push your application.

I'll save on the details, except to say that this is the code we used when we transferred to RackSpace:

server {
   listen [ip];
   root /var/www/viewgit;

   server_name git.domain.com;
      location ~ \.php$ {
        include fastcgi_params;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        fastcgi_pass 127.0.0.1:9001;
      }
      location ~ ^projects/.*/(HEAD|info/refs|objects/info/.*|git-upload-pack)$   {
        root /var/www/viewgit/projects;
        include fastcgi_params;
        fastcgi_param SCRIPT_FILENAME   /usr/lib/git-core/git-http-backend;
        fastcgi_param PATH_INFO         $uri;
        fastcgi_param GIT_PROJECT_ROOT  /var/www/viewgit/projects;
        fastcgi_param GIT_HTTP_EXPORT_ALL "";
        fastcgi_pass unix:/var/run/fcgiwrap.socket;
     }
}

This will allow you to add another remote repo to your local app code:

git add remote X http://git.yourdomain.com/your_code.git

Doing this will give you a direct way to push your code to your new server. I won't go into post-receive hooks etc, and how they should be used to get it working properly.

--

Database

The database is slightly trickier, but not insurmountable.

The way to do it is to use pg:backups, which essentially creates a "dump" of your database, allowing you to load it and deploy it elsewhere.

This is much harder to do with PGSQL > MYSQL. But since Heroku uses Amazon's own databases anyway, I don't think you'd have an issue.

You'll be best following the instructions to download your heroku DB here.

Once you've downloaded the db, you'll then have to zip it up and upload it into the AWS service. As mentioned, I don't have massive experience with this, so I'll just say that if you need any help, I'll write an update for you.