Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How can AWS CloudFormation pull code from Git?

I have source-code that lives in a Git repo (GitHub). I have several branches for different environments (e.g. develop, production) and I prefer to manage releases inside the GitHub interface using protected branches.

I want a push to a branch to trigger tests and deployment.

However, I also use CloudFormation to deploy AWS services in a reproducible way. My problem is in interfacing CloudFormation with my Git process.

For example, AWS Lambda functions are described in CloudFormation templates like this:

{
  "Type" : "AWS::Lambda::Function",
  "Properties" : {
    "Code": "source code here"
  }
}

... where Code is "Amazon Simple Storage Service (Amazon S3) bucket or specify your source code as inline text. " (Docs)

This means I need to do a manual step after deploying my CloudFormation template:

  • Checkout the latest AWS Lambda code from $BRANCH at $REPO
  • Run any tests
  • Run my build & package script
  • Upload the code to AWS Lambda

(This can be done in a CI provider, but then I still have to click "Rebuild" on each repo)

What I would rather do is define my CI pipeline inside the CloudFormation template. This should be possible using EC2 etc, but I don't know how. The Git repo URL would then be a parameter of the CloudFormation template.

How do I define Git hooks, build steps and deployment in a CloudFormation template? The steps should also run as part of a fresh CloudFormation deployment.

like image 688
sdgfsdh Avatar asked Jan 04 '18 16:01

sdgfsdh


People also ask

How does AWS CloudFormation work?

CloudFormation creates a bucket for each region in which you upload a template file. The buckets are accessible to anyone with Amazon Simple Storage Service (Amazon S3) permissions in your AWS account. If a bucket created by CloudFormation is already present, the template is added to that bucket.

Can CloudFormation run a script?

For those learning AWS/AWS CLI, CloudFormation is a tool for building infrastructure with AWS. Here is a very simple document on how to use CloudFormation to build an AWS EC2 Linux instance and execute a bash script from CloudFormation against the newly created Linux instance.

Does CloudFormation allow custom scripts?

Custom resources enable you to write custom provisioning logic in templates that AWS CloudFormation runs anytime you create, update (if you changed the custom resource), or delete stacks.


1 Answers

Normally, you would not build as part of a CloudFormation deployment. The builds happen before CloudFormation deployment happens.

So, as part of your Ci pipeline (not in CloudFormation), you would:

  1. Update your git repo
  2. Trigger a build of your new code
  3. Upload your artifacts (packages) to S3, never overwriting old artifacts (for example, upload to /artifacts/{build number}/MyLambda.zip
  4. Trigger a CloudFormation deployment

As part of step 4, you would pass the location of the artifacts created in step 3 into your CloudFormation stack as parameters and use those parameters to build your Lambda source code location.

Also you should utilize Lambda environment variables in your CloudFormation template to dictate dev/prod/staging parameters, and don't have them hard coded into your Lambda package. This allows you to re-use the same build package between dev/prod/staging.

like image 160
Matt Houser Avatar answered Oct 12 '22 15:10

Matt Houser