Update cloudfront configuration using awscli

Question

I would like to edit/update my CloudFront distribution with awscli.

I'm using latest cli version:

aws-cli/1.11.56 Python/2.7.10 Darwin/16.4.0 botocore/1.5.19

To use cloudfront features in awscli you need to add this to your aws config file:

[preview]
cloudfront = true

I'm getting config of the distribution that I'd like to modify:

aws cloudfront get-distribution-config --id FOO_BAR_ID > cf_config.json

Looks like it worked as expected. Config looks ok for me. Now I'm trying to reconfigure my CF distribution with the same config.

aws cloudfront update-distribution --distribution-config file://cf_config.json --id FOO_BAR_ID

and I'm getting:

Parameter validation failed:
Missing required parameter in DistributionConfig: "CallerReference"
Missing required parameter in DistributionConfig: "Origins"
Missing required parameter in DistributionConfig: "DefaultCacheBehavior"
Missing required parameter in DistributionConfig: "Comment"
Missing required parameter in DistributionConfig: "Enabled"
Unknown parameter in DistributionConfig: "ETag", must be one of: CallerReference, Aliases, DefaultRootObject, Origins, DefaultCacheBehavior, CacheBehaviors, CustomErrorResponses, Comment, Logging, PriceClass, Enabled, ViewerCertificate, Restrictions, WebACLId, HttpVersion, IsIPV6Enabled
Unknown parameter in DistributionConfig: "DistributionConfig", must be one of: CallerReference, Aliases, DefaultRootObject, Origins, DefaultCacheBehavior, CacheBehaviors, CustomErrorResponses, Comment, Logging, PriceClass, Enabled, ViewerCertificate, Restrictions, WebACLId, HttpVersion, IsIPV6Enabled

What is the right way to reconfigure CF using awscli?

Answer 1

@usterk's answer is correct, but it took me another 3 hours to get to the script that I needed. Here, I am sharing it.

My case: CI/CD using S3/CloudFront with manual artifact versioning

I am hosting a static website (SSG) in S3, and I want it to be served by CloudFront. The website gets frequent updates in terms of its code (not just the content) and I want to store all the versions of the website in S3 (just like all the artifacts or docker images) and update CloudFront to point to a new version, right after a new version is pushed to S3.

I know that there is "file versioning" in S3, but this old-school format for keeping all versions of the assets helps with analyzing the assets as well as easy roll-backs.

My configs

• After building the assets (JS, CSS, etc), the new files are uploaded to S3 in a folder like s3://<mybucket-name>/artifacts/<version-id>
• In CloudFront I have a Distribution for www website. Route53 for www.domain.com is pointing to it.
• In that Distribution I have several Origins (e.g. one to send /api path to ELB.)
• The Origin that matter here is www which has its OriginPath pointing to /artifacts/<version-id>.

Workflow

• After S3 sync is done via AWS CLI, I need to update CloudFront's configs for that www Origin's OriginPath value to point to the new path in S3.
• I also need to initiate an invalidation on the Distribution so CloudFront picks up the new files internally (between S3 and it)

The Task

As @usterk and @BrianLeishman pointed out, the only CLI command for this job is update-distribution which per the documentation, requires the ENTIRE CONFIGURATION of the distribution to REPLACE it. So, there is no command to partially update just one field in the config.

To achieve this, one must first get the current distribution-config, then extract the "DistributionConfig" component, then update the fields it takes, and finally, put it back in the proper format with a proper verification token.

Note that what the "update" command needs is a "subset" of what "get" gives back. So parsing JSON via jq is inevitable.

The Bash Script

The following script that I came up with, does the job for me:

# 0) You need to set the followings for your case
CLOUDFRONT_DISTRIBUTION_ID="EABCDEF12345ABCD"
NEW_ORIGIN_PATH="/art/0.0.9"
CLOUDFRONT_ORIGIN_ID="E1A2B3C4D5E6F"

DIST_CONFIG_OLD_FILENAME="dist-config.json" # a temp file, which will be removed later
DIST_CONFIG_NEW_FILENAME="dist-config2.json" # a temp file, which will be removed later

# 1) Get the current config, entirely, and put it in a file
aws cloudfront get-distribution --id $CLOUDFRONT_DISTRIBUTION_ID > $DIST_CONFIG_OLD_FILENAME

# 2) Extract the Etag which we need this later for update
Etag=`cat $DIST_CONFIG_OLD_FILENAME | jq '.ETag' | tr -d \"`

# 3) Modify the config as wished, for me I used `jq` extensively to update the "OriginPath" of the desired "originId"
cat $DIST_CONFIG_OLD_FILENAME | jq \
    --arg targetOriginId $CLOUDFRONT_ORIGIN_ID \
    --arg newOriginPath $NEW_ORIGIN_PATH \
    '.Distribution.DistributionConfig | .Origins.Items = (.Origins.Items | map(if (.Id == $targetOriginId) then (.OriginPath = $newOriginPath) else . end))' \
    > $DIST_CONFIG_NEW_FILENAME

# 4) Update the distribution with the new file
aws cloudfront update-distribution --id $CLOUDFRONT_DISTRIBUTION_ID \
    --distribution-config "file://${DIST_CONFIG_NEW_FILENAME}" \
    --if-match $Etag \
    > /dev/null

# 5) Invalidate the distribution to pick up the changes
aws cloudfront create-invalidation --distribution-id $CLOUDFRONT_DISTRIBUTION_ID --paths "/*"

# 6) Clean up
rm -f $DIST_CONFIG_OLD_FILENAME $DIST_CONFIG_NEW_FILENAME

Final Note: IAM Access

The user that performs these needs IAM access to the Get, Invalidate, and Update actions on the Distribution in CloudFront. Here is the Policy that gives that:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "cloudfront:GetDistribution",
                "cloudfront:UpdateDistribution",
                "cloudfront:CreateInvalidation"
            ],
            "Resource": "arn:aws:cloudfront::<ACCOUNT_ID>:distribution/<DISTRIBUTION_ID>
        }
    ]
}

Answer 2

You have to edit cf_config.json before using it with update-distribution and remove

{
    "ETag": "ETag_Value",
    "DistributionConfig":

from the beginning of the file and last

}

from the end of file.

Then use this command with the right id and ETag value that was removed from cf_config.json

aws cloudfront update-distribution --distribution-config file://cf_config.json --id FOO_BAR_ID --if-match ETag_Value