How to handle multiple environments in CodePipeline?

Question

I'm using code pipeline to deploy my infrastructure and I would like to be able to deploy it in different environments (dev, staging, prod,...).

I currently have a buildspec.yml file containing some "pip install" instructions and the "aws cloudformation package" command. I also created 2 pipelines, one for production and another for development pointing to 2 different branches on github. The problem I have is that since in both branches the files contains similar resources, I have name collision on the S3 buckets for example.

When using the AWS CLI and cloudformation to create or update a stack you can pass parameters using the --parameters option. I would like to do something similar in the 2 pipelines I've created.

What would be the best solution to tackle this issue?

The final goal is to automate the deployment of our infrastructure. Our infrastructure is composed of Users, KMS keys, Lamdbas (in python), Groups and Buckets.

I've created two pipelines following the tutorial: http://docs.aws.amazon.com/lambda/latest/dg/automating-deployment.html

The first pipeline is linked to the master branch of the repo containing the code and the second one to the staging branch. My goal is to automate the deployment of the master branch in the production environment using the first pipeline and the staging branch in the staging environment using the second one.

My buildspec.yml file look like:

version: 0.1
phases:
    install:
        commands:
            - pip install requests -t .
            - pip install simplejson -t .
            - pip install Image -t .
            - aws cloudformation package --template-file image_processing_sam.yml --s3-bucket package-bucket --output-template-file new_image_processing_sam.yml
artifacts:
    type: zip
    files:
        - new_image_processing_sam.yml

The image_processing_sam.yml file look like:

AWSTemplateFormatVersion: "2010-09-09"
Transform: "AWS::Serverless-2016-10-31"
Description: Create a thumbnail for an image uploaded to S3
Resources:

  ThumbnailFunction:
    Type: "AWS::Serverless::Function"
    Properties:
      Role: !GetAtt LambdaExecutionRole.Arn
      Handler: create_thumbnail.handler
      Runtime: python2.7
      Timeout: 30
      Description: "A function computing the thumbnail for an image."

  LambdaSecretEncryptionKey:
    Type: "AWS::KMS::Key"
    Properties:
      Description: "A key used to encrypt secrets used in the Lambda functions"
      Enabled: True
      EnableKeyRotation: False
      KeyPolicy:
        Version: "2012-10-17"
        Id: "lambda-secret-encryption-key"
        Statement:
          -
            Sid: "Allow administration of the key"
            Effect: "Allow"
            Principal:
              AWS: "arn:aws:iam::xxxxxxxxxxxxx:role/cloudformation-lambda-execution-role"
            Action:
              - "kms:Create*"
              - "kms:Describe*"
              - "kms:Enable*"
              - "kms:List*"
              - "kms:Put*"
              - "kms:Update*"
              - "kms:Revoke*"
              - "kms:Disable*"
              - "kms:Get*"
              - "kms:Delete*"
              - "kms:ScheduleKeyDeletion"
              - "kms:CancelKeyDeletion"
            Resource: "*"
          -
            Sid: "Allow use of the key"
            Effect: "Allow"
            Principal:
              AWS:
                - !GetAtt LambdaExecutionRole.Arn
            Action:
              - "kms:Encrypt"
              - "kms:Decrypt"
              - "kms:ReEncrypt*"
              - "kms:GenerateDataKey*"
              - "kms:DescribeKey"
            Resource: "*"

  LambdaExecutionRole:
    Type: "AWS::IAM::Role"
    Properties:
      RoleName: "LambdaExecutionRole"
      AssumeRolePolicyDocument:
        Version: "2012-10-17"
        Statement:
        - Effect: Allow
          Principal:
            Service:
            - "lambda.amazonaws.com"
          Action:
          - "sts:AssumeRole"
      Policies:
        -
          PolicyName: LambdaKMS
          PolicyDocument:
            Version: "2012-10-17"
            Statement:
              -
                Effect: "Allow"
                Action:
                  - "kms:Decrypt"
                Resource: "*"
              -
                Effect: "Allow"
                Action:
                  - "lambda:InvokeFunction"
                Resource: "*"
      ManagedPolicyArns:
      - "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"

  UserGroup:
      Type: "AWS::IAM::Group"

  LambdaTriggerUser:
    Type: "AWS::IAM::User"
    Properties:
      UserName: "LambdaTriggerUser"

  LambdaTriggerUserKeys:
    Type: "AWS::IAM::AccessKey"
    Properties:
      UserName:
        Ref: LambdaTriggerUser

  Users:
    Type: "AWS::IAM::UserToGroupAddition"
    Properties:
      GroupName:
        Ref: UserGroup
      Users:
        - Ref: LambdaTriggerUser

  Policies:
    Type: "AWS::IAM::Policy"
    Properties:
      PolicyName: UserPolicy
      PolicyDocument:
        Statement:
          -
            Effect: "Allow"
            Action:
              - "lambda:InvokeFunction"
            Resource:
              - !GetAtt DispatcherFunction.Arn
      Groups:
        - Ref: UserGroup

  PackageBucket:
    Type: "AWS::S3::Bucket"
    Properties:
      BucketName: "package-bucket"
      VersioningConfiguration:
        Status: "Enabled"

Outputs:
  LambdaTriggerUserAccessKey:
    Value:
      Ref: "LambdaTriggerUserKeys"
    Description: "AWSAccessKeyId of LambdaTriggerUser"

  LambdaTriggerUserSecretKey:
    Value: !GetAtt LambdaTriggerUserKeys.SecretAccessKey
    Description: "AWSSecretKey of LambdaTriggerUser"

I've added a deploy action in both pipelines to execute the change set computed during the beta action.

The first pipeline works like a charm and does everything I expect it to do. Each time I push code in the master branch, it's deployed.

The issue I'm facing is that when I push code in the staging branch, everything works in the pipeline until the deploy action is reached. The deploy action try to create a new stack but since it's exactly the same buildspec.yml and the image_processing_sam.yml that is processed, I reach name collision like below.

package-bucket already exists in stack arn:aws:cloudformation:eu-west-1:xxxxxxxxxxxx:stack/master/xxxxxx-xxxx-xxx-xxxx
LambdaTriggerUser already exists in stack arn:aws:cloudformation:eu-west-1:xxxxxxxxxxxx:stack/master/xxxxxx-xxxx-xxx-xxxx
LambdaExecutionRole already exists in stack arn:aws:cloudformation:eu-west-1:xxxxxxxxxxxx:stack/master/xxxxxx-xxxx-xxx-xxxx
...

Is there a way to parameterize the buildspec.yml to be able to add a suffix to the name of the resources in the image_processing_sam.yml? Any other idea to achieve this is welcome.

Best regards.

Answer 1

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/continuous-delivery-codepipeline-basic-walkthrough.html

Template configuration files are applied to the CloudFormation in the CodePipeline via a parameter file like this:

{
  "Parameters" : {
    "DBName" : "TestWordPressDB",
    "DBPassword" : "TestDBRootPassword",
    "DBRootPassword" : "TestDBRootPassword",
    "DBUser" : "TestDBuser",    
    "KeyName" : "TestEC2KeyName"
    }
}

Place these files in the root of your repo and they can be referenced in at least 2 ways.

In your CodePipeline CloudFormation:

Configuration:
    ActionMode: REPLACE_ON_FAILURE
    RoleArn: !GetAtt [CFNRole, Arn]
    StackName: !Ref TestStackName
    TemplateConfiguration: !Sub "TemplateSource::${TestStackConfig}"
    TemplatePath: !Sub "TemplateSource::${TemplateFileName}"

Or in the console in the Template configuration field:

It is worth noting the config file format is different from CloudFormation via cli using

-- parameters

--parameters uses this format:

[
  {
    "ParameterKey": "team",
    "ParameterValue": "AD-Student Life Applications"
  },
  {
    "ParameterKey": "env",
    "ParameterValue": "dev"
  },
  {
    "ParameterKey": "dataSensitivity",
    "ParameterValue": "public"
  },
  {
    "ParameterKey": "app",
    "ParameterValue": "events-list-test"
  }
]

CodePipeline Cloudformation template configuration files use this format:

{
  "Parameters" : {
    "DBName" : "TestWordPressDB",
    "DBPassword" : "TestDBRootPassword",
    "DBRootPassword" : "TestDBRootPassword",
    "DBUser" : "TestDBuser",    
    "KeyName" : "TestEC2KeyName"
  }
}

Answer 2

Check the Eric Nord's answer. It is the one you are looking for.

---

I asked the question on the AWS forum as well here.

Here is the solution provided by AWS:

Hi,

If your goal is to have different bucket names for staging and master, then another option is to use CloudFormation parameters.

When editing an existing pipeline if you edit an action you can expand the "Advanced" panel and enter parameter overrides to specify a different bucket prefix for each stage. You can also enter parameters as a separate .json file in your artifact.

There's more details on doing that here: http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/continuous-delivery-codepipeline-parameter-override-functions.html

Here's a full walk through with a different stack configuration for test and production: http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/continuous-delivery-codepipeline-basic-walkthrough.html

• Tim.

You should definitely follow the documentation provided. Here's the solution I came up with below.

---

Here is my own solution that I wasn't satisfied with.

I've added a script running at build time and modifying my template given the ARN of the CodeBuild agent building the project.

I've added "BRANCH_NAME" where naming collision can occur. The image_processing_sam.yml is now:

AWSTemplateFormatVersion: "2010-09-09"
Transform: "AWS::Serverless-2016-10-31"
Description: Create a thumbnail for an image uploaded to S3
Resources:

  ThumbnailFunction:
    Type: "AWS::Serverless::Function"
    Properties:
      Role: !GetAtt LambdaExecutionRole.Arn
      Handler: create_thumbnail.handler
      Runtime: python2.7
      Timeout: 30
      Description: "A function computing the thumbnail for an image."

  LambdaSecretEncryptionKey:
    Type: "AWS::KMS::Key"
    Properties:
      Description: "A key used to encrypt secrets used in the Lambda functions"
      Enabled: True
      EnableKeyRotation: False
      KeyPolicy:
        Version: "2012-10-17"
        Id: "lambda-secret-encryption-keyBRANCH_NAME"
        Statement:
          -
            Sid: "Allow administration of the key"
            Effect: "Allow"
            Principal:
              AWS: "arn:aws:iam::xxxxxxxxxxxxx:role/cloudformation-lambda-execution-role"
            Action:
              - "kms:Create*"
              - "kms:Describe*"
              - "kms:Enable*"
              - "kms:List*"
              - "kms:Put*"
              - "kms:Update*"
              - "kms:Revoke*"
              - "kms:Disable*"
              - "kms:Get*"
              - "kms:Delete*"
              - "kms:ScheduleKeyDeletion"
              - "kms:CancelKeyDeletion"
            Resource: "*"
          -
            Sid: "Allow use of the key"
            Effect: "Allow"
            Principal:
              AWS:
                - !GetAtt LambdaExecutionRole.Arn
            Action:
              - "kms:Encrypt"
              - "kms:Decrypt"
              - "kms:ReEncrypt*"
              - "kms:GenerateDataKey*"
              - "kms:DescribeKey"
            Resource: "*"

  LambdaExecutionRole:
    Type: "AWS::IAM::Role"
    Properties:
      RoleName: "LambdaExecutionRoleBRANCH_NAME"
      AssumeRolePolicyDocument:
        Version: "2012-10-17"
        Statement:
        - Effect: Allow
          Principal:
            Service:
            - "lambda.amazonaws.com"
          Action:
          - "sts:AssumeRole"
      Policies:
        -
          PolicyName: LambdaKMSBRANCH_NAME
          PolicyDocument:
            Version: "2012-10-17"
            Statement:
              -
                Effect: "Allow"
                Action:
                  - "kms:Decrypt"
                Resource: "*"
              -
                Effect: "Allow"
                Action:
                  - "lambda:InvokeFunction"
                Resource: "*"
      ManagedPolicyArns:
      - "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"

  UserGroup:
      Type: "AWS::IAM::Group"

  LambdaTriggerUser:
    Type: "AWS::IAM::User"
    Properties:
      UserName: "LambdaTriggerUserBRANCH_NAME"

  LambdaTriggerUserKeys:
    Type: "AWS::IAM::AccessKey"
    Properties:
      UserName:
        Ref: LambdaTriggerUser

  Users:
    Type: "AWS::IAM::UserToGroupAddition"
    Properties:
      GroupName:
        Ref: UserGroup
      Users:
        - Ref: LambdaTriggerUser

  Policies:
    Type: "AWS::IAM::Policy"
    Properties:
      PolicyName: UserPolicyBRANCH_NAME
      PolicyDocument:
        Statement:
          -
            Effect: "Allow"
            Action:
              - "lambda:InvokeFunction"
            Resource:
              - !GetAtt DispatcherFunction.Arn
      Groups:
        - Ref: UserGroup

  PackageBucket:
    Type: "AWS::S3::Bucket"
    Properties:
      BucketName: "package-bucketBRANCH_NAME"
      VersioningConfiguration:
        Status: "Enabled"

Outputs:
  LambdaTriggerUserAccessKey:
    Value:
      Ref: "LambdaTriggerUserKeys"
    Description: "AWSAccessKeyId of LambdaTriggerUser"

  LambdaTriggerUserSecretKey:
    Value: !GetAtt LambdaTriggerUserKeys.SecretAccessKey
    Description: "AWSSecretKey of LambdaTriggerUser"

The script.sh replacing the "BRANCH_NAME" in the template is:

#!/bin/bash
echo $CODEBUILD_AGENT_ENV_CODEBUILD_BUILD_ARN
if [[ "$CODEBUILD_AGENT_ENV_CODEBUILD_BUILD_ARN" == *"master"* ]]; then
    sed "s/BRANCH_NAME//g" image_processing_sam.yml > generated_image_processing_sam.yml;
fi
if [[ "$CODEBUILD_AGENT_ENV_CODEBUILD_BUILD_ARN" == *"staging"* ]]; then
    sed "s/BRANCH_NAME/staging/g" image_processing_sam.yml > generated_image_processing_sam.yml;
fi

The buildspec.yml is now:

version: 0.1
phases:
    install:
        commands:
            # Install required module for python
            - pip install requests -t .
            - pip install simplejson -t .
            - pip install Image -t .
            - bash ./script.sh
            # To be able to see any issue in the generated template
            - cat generated_image_processing_sam.yml
            # Package the generated cloudformation template in order to deploy
            - aws cloudformation package --template-file generated_image_processing_sam.yml --s3-bucket piximate-package-bucket --output-template-file new_image_processing_sam.yml
artifacts:
    type: zip
    files:
        - new_image_processing_sam.yml

I hope it can somehow help you. I would be glad if anyone can provide any improvement or documentation that could help.