How should I migrate (update DB schema) my DB for my AWS Serverless application

Question

How should I be running my DB migrations in a AWS Serverless application? In a traditional NodeJS app, I usually have npm start run sequelize db:migrate first. But with Lambda how should I do that?

My DB will be in a private subnet. Was wondering if CodeBuild will be able to do it? Was also considering to have a Lambda function run migration ... not sure if its recommended tho.

Answer 1

There are a number of ways to achieve this. You actually are kind of on the right track with CodeBuild, at least there shouldn't be anything wrong with taking that approach.

Since your DB is in a private subnet, you will need to configure CodeBuild to access your VPC. Once you have that configured, it's a simple matter of allowing access from the CodeBuild security group to your database.

You might want to setup this whole thing as a CodePipeline. You might even set it up with multiple buildspec files for different runs of CodeBuild. That way you can have a CodePipeline that looks like:

Source -> CodeBuild (test) -> Approval -> CodeBuild (migrations) -> Lambda

Theoretically, you could also create a Lambda function that does the migration, and trigger that as needed. If the migrations take a long time, you could also use AWS Batch to run them. But using CodeBuild as part of a deployment pipeline makes a lot of sense.

Answer 2

Lambda might not be a right tool for this task because of short runtime.

You are better of using a custom script run on the CodeBuild. And having a sequential CodeBuild tasks in your Codepipeline where first codebuild will complete the migration and on-complete of the first CodeBuild, you can execute the new CodeBuild which will deploy your lambdas. Just in case if your DB migration fails, you can exit the CodePipeLine.

Your CodePipeLine will looks like this.

pre_build:    
    commands:
     - DB migration command
    finally:
     - CleanUp Command 
build:
     commands:
     - Deploy lambdas command
     finally:
     - Cleanup command

Answer 3

Both approaches (lambda and codebuild) are ok, it depends on your Continuous Deployment/Integration flow. For example, if you need to run those migrations on several environments, Codebuild would fit better.

If you don't have a CI/CD mechanism, you can just run it on a lambda, as it's very flexible in terms of memory (you just need to be careful in the maximum execution time), or use an already made package like this (this is a suggestion and depends on your db).

As a last opinion, if your process is really heavy and/or needs to do a lot of read/write operations, you could also try running it on an AWS ECS instance which will scale when you run the migrations and after finishing, return to its minimum size defined.