How can AWS CloudFormation pull code from Git?

Question

I have source-code that lives in a Git repo (GitHub). I have several branches for different environments (e.g. develop, production) and I prefer to manage releases inside the GitHub interface using protected branches.

I want a push to a branch to trigger tests and deployment.

However, I also use CloudFormation to deploy AWS services in a reproducible way. My problem is in interfacing CloudFormation with my Git process.

For example, AWS Lambda functions are described in CloudFormation templates like this:

{
  "Type" : "AWS::Lambda::Function",
  "Properties" : {
    "Code": "source code here"
  }
}

... where Code is "Amazon Simple Storage Service (Amazon S3) bucket or specify your source code as inline text. " (Docs)

This means I need to do a manual step after deploying my CloudFormation template:

• Checkout the latest AWS Lambda code from $BRANCH at $REPO
• Run any tests
• Run my build & package script
• Upload the code to AWS Lambda

(This can be done in a CI provider, but then I still have to click "Rebuild" on each repo)

What I would rather do is define my CI pipeline inside the CloudFormation template. This should be possible using EC2 etc, but I don't know how. The Git repo URL would then be a parameter of the CloudFormation template.

How do I define Git hooks, build steps and deployment in a CloudFormation template? The steps should also run as part of a fresh CloudFormation deployment.

Answer 1

Normally, you would not build as part of a CloudFormation deployment. The builds happen before CloudFormation deployment happens.

So, as part of your Ci pipeline (not in CloudFormation), you would:

1. Update your git repo
2. Trigger a build of your new code
3. Upload your artifacts (packages) to S3, never overwriting old artifacts (for example, upload to /artifacts/{build number}/MyLambda.zip
4. Trigger a CloudFormation deployment

As part of step 4, you would pass the location of the artifacts created in step 3 into your CloudFormation stack as parameters and use those parameters to build your Lambda source code location.

Also you should utilize Lambda environment variables in your CloudFormation template to dictate dev/prod/staging parameters, and don't have them hard coded into your Lambda package. This allows you to re-use the same build package between dev/prod/staging.