Is there a way to unit test AWS Cloudformation template

Question

When we say that cloudformation is 'Infrastructure as Code', the next question that immediately comes to mind is how can this code be tested. Can we do some sort of basic unit test of this code

And I am discounting the cloudformation validation because that just is a way of doing syntactic validation, and that I can do with any other free JSON/YAML validator.

I am more inclined towards some sort of functional validation, possibly testing that I have defined all the variables that are used as references. Possibly testing that whatever properties I am using are actually supported ones for that component

Not expected that it should test if the permissions are correct or that I have not exhausted my limits. But atleast something beyond the basic JSON/YAML syntax validation

Answer 1

Here's a breakdown of how several methods of testing software can be applied to CloudFormation templates/stacks:

Linting

For linting (checking CloudFormation-template code for syntax/grammar correctness), you can use the ValidateTemplate API to check basic template structure, and the CreateChangeSet API to verify your Resource properties in more detail.

• Note that ValidateTemplate performs a much more thorough check than a simple JSON/YAML syntax checker- it validates correct Template Anatomy, correct syntax/usage of Intrinsic Functions, and correct resolution of all Ref values.
• ValidateTemplate checks basic CloudFormation syntax, but doesn't verify your template's Resources against specific property schemas. For checking the structure of your template's Parameters, Resources and Properties against AWS Resource types, CreateChangeSet should return an error if any parameters or resource properties are not well-formed.

Unit testing

Performing unit testing first requires an answer to the question: what is the smallest self-contained unit of functionality that can/should be tested? For CloudFormation, I believe that the smallest testable unit is the Resource.

The official AWS Resource Types are supported/maintained by AWS (and are proprietary implementations anyway) so don't require any additional unit tests written by end-user developers.

However, your own Custom Resources could and should be unit-tested. This can be done using a suitable testing framework in the implementation's own language (e.g., for Lambda-backed Custom Resources, perhaps a library like lambda-tester would be a good starting point).

Integration testing

This is the most important and relevant type of testing for CloudFormation stacks (which mostly serve to tie various Resources together into an integrated application), and also the type that could use more refinement and best-practice development. Here are some initial ideas on how to integration-test CloudFormation code by actually creating/updating full stacks containing real AWS resources:

• Using a scripting language, perform a CloudFormation stack creation using the language's AWS SDK. Design the template to return Stack Outputs reflecting behavior that you want to test. After the stack is created by the scripting language, compare the stack outputs against expected values (and then optionally delete the stack afterwards in a cleanup process).
• Use AWS::CloudFormation::WaitCondition resources to represent successful tests/assertions, so that a successful stack creation indicates a successful integration-test run, and a failed stack creation indicates a failed integration-test run.

---

Beyond CloudFormation, one interesting tool worth mentioning in the space of testing infrastructure-as-code is kitchen-terraform, a set of plugins for Test Kitchen which allow you to write fully-automated integration test suites for Terraform modules. A similar integration-testing harness could eventually be built for CloudFormation, but doesn't exist yet.

Answer 2

This tool “cfn-nag” parses a collection of CloudFormation templates and applies rules to find code patterns that could lead to insecure infrastructure.  The results of the tool include the logical resource identifiers for violating resources and an explanation of what rule has been violated. Further Reading: https://stelligent.com/2016/04/07/finding-security-problems-early-in-the-development-process-of-a-cloudformation-template-with-cfn-nag/

While there are quite a number of particular rules the tool will attempt to match, the rough categories are:

IAM and resource policies (S3 Bucket, SQS, etc.) Matches policies that are overly permissive in some way (e.g. wildcards in actions or principals)

Security Group ingress and egress rules Matches rules that are overly liberal (e.g. an ingress rule open to 0.0.0.0/0, port range 1-65535 is open)

Access Logs Looks for access logs that are not enabled for applicable resources (e.g. Elastic Load Balancers and CloudFront Distributions)

Encryption (Server-side) encryption that is not enabled or enforced for applicable resources (e.g. EBS volumes or for PutObject calls on an S3 bucket)

Answer 3

New tool is on the market now. Test all the CloudFormation things! (with TaskCat)

What is taskcat?

taskcat is a tool that tests AWS CloudFormation templates. It deploys your AWS CloudFormation template in multiple AWS Regions and generates a report with a pass/fail grade for each region. You can specify the regions and number of Availability Zones you want to include in the test, and pass in parameter values from your AWS CloudFormation template. taskcat is implemented as a Python class that you import, instantiate, and run.

usage

follow this document : https://aws.amazon.com/blogs/infrastructure-and-automation/up-your-aws-cloudformation-testing-game-using-taskcat/

notes

1. taskcat can't read AWS_PROFILE environment variable. you need define the profile in part of general if it is not default profile.

general:
  auth:
    default: dev-account

Ref: https://github.com/aws-quickstart/taskcat/issues/434