Serverless CloudFormation Linting in AWS CodePipeline

Stelligent’s cfn-nag-pipeline is a serverless open source tool that generates an AWS Lambda function that can be used as an Invoke action in AWS CodePipeline and has been made available on the AWS Serverless Application Repository (SAR). You can use it to automatically detect security vulnerabilities in CloudFormation templates as part of your deployment pipelines. The automated provisioning is implemented in AWS CloudFormation and the automated execution of cfn_nag is implemented in Ruby and called by AWS Lambda (via JRuby).
In February 2018, AWS announced the general availability of the AWS SAR – which enables you to discover, deploy, and publish serverless applications. The cfn-nag-pipeline integration has been deployed to the AWS SAR here.
Stelligent cfn_nag is an open source command-line tool that performs static analysis of AWS CloudFormation templates. With cfn_nag, you can:

• Perform static code analysis of AWS CloudFormation
• Block undesirable resource specifications
• Perform Proactive preventative control – stop before creating resources
• Integrate into a deployment pipeline

Here are some examples of the types of checks cfn_nag can perform:

• EC2 Instance Security Groups with ingress of 0.0.0.0/0
• IAM Permissions given to all (*) resources or all (*) actions
• EBS volumes for full disk encryption
• Access logging

You can also extend cfn_nag’s capabilities through custom rules. To learn more on custom rule development, go here.
You can read more about cfn_nag in this blog post and README.
In this blog post, you’ll learn the steps for deploying this solution using the AWS SAR or through the AWS Lambda console. Once you’ve successfully deployed the solution, you can configure it in the CodePipeline console or by using its configuration output.
By utilizing the serverless capabilities of AWS Lambda, you reduce the amount of upfront configuration to get cfn_nag running on AWS CodePipeline. With this serverless implementation, you don’t need to download, install, and configure cfn_nag: it just works.

Locating the cfn-nag-pipeline application in the AWS SAR

There are two ways to locate the cfn-nag-pipeline application in the AWS SAR. You can either search for the application in the AWS Lambda Console or go directly to the application page that SAR creates for each application.
Option 1:

• Go to the AWS Lambda Console
• Click Create function
• Select the Serverless Application Repository radio button
• Type cfn-nag-pipeline in the search applications by name, description, or labels text box
• As results are displayed, select the cfn-nag-pipeline link

Option 2:

• Go to the cfn-nag-pipeline solution in the AWS SAR

Deployment Steps

Once you have located the cfn-nag-pipeline application in AWS SAR or AWS Lambda, follow these steps for deploying the application:

1. In the Configure application parameters section, type a unique name for the stack.
2. Enter * for the PipelineBucketName parameter
3. Scroll down and click Deploy. You’ll see resources being created by the CloudFormation stack. It will take approximately 2 minutes to complete the stack creation. You’ll see a message that states “Application successfully deployed”.

1. Complete the remaining steps by going to the Configure IAM Role Policies section in this blog post.

Configure IAM Role Policies

Due to a bug with SAM policy templates (#389), you will need to manually update the IAM role that the Lambda function assumes. The role should be named something likeaws-serverless-repository-cfn-n-CfnNagFunctionRole-XXXXXXXX. Update the inline policy named CfnNagFunctionRolePolicy0 to set the Resource to * as shown below:

1. Go to IAM Roles
2. Select the IAM role name
3. Click Add inline policy
4. Click the JSON tab and paste the content below:
5. Click though the remaining steps to name and save the IAM policy

 {
   "Version": "2012-10-17",
   "Statement": [
     {
       "Effect": "Allow",
       "Action": [
         "codepipeline:PutJobFailureResult",
         "codepipeline:PutJobSuccessResult"
       ],
       "Resource": "*"
     }
  ]
}

Add the cfn-nag-pipeline Action using the CodePipeline Console

Now that you’ve configured the Lambda function and IAM Policy, you’re ready to execute it from within one of your pipelines in AWS CodePipeline. To do this, you’ll add a new Action to AWS CodePipeline by following the steps below:

1. From the CodePipeline console, click on an existing pipeline (or create a new pipeline) and ensure there is a Source action for a repository that contains CloudFormation templates.
2. Add a downstream Invoke Action Category with AWS Lambda as the Provider. Provide an Action name.
3. Select cfn-nag-pipeline for the Function name
4. Select the glob for CloudFormation templates in the User parameters field: e.g. cfn/*.yml
5. In the Input artifacts field, select the name of the input artifact from the repository in the Input artifacts field.

Figure 1 shows a screenshot of steps 2-5.

Figure 1 – Add an Invoke action for cfn-nag-pipeline

1. Click the Add action button
2. Click the Save pipeline changes button.

Troubleshoot and Resolve cfn_nag Failures

Run an invocation of the pipeline you just updated:
1) Click the Release changes button.
2) Once the pipeline gets to the cfn-nag-pipeline action, if any cfn_nag rules are violated, the action was fail – as shown in Figure 2. Click on the Details link.
 

Figure 2 – CodePipeline cfn-nag Action Failure

3) A window displays showing the number of cfn_nag failures and warnings. Click on the Link to execution details – as shown in Figure 3.

Figure 3 – CodePipeline cfn-nag Action Detail

4) This brings you to CloudWatch Logs. Click on the most recent Log Stream and scroll down to find the failures. In Figure 4, you see that I have four IAM roles that are using * in its permissions policy which is a built-in cfn_nag rule violation.

Figure 4 – CodePipeline cfn-nag Action CloudWatch Logs

5) Modify your CloudFormation template(s) to ensure that it passes all the cfn_nag rules.
In my case, I modified my CloudFormation template to ensure that I specify the actions these IAM roles can use and committed the code to my version-control repository. Once CodePipeline discovered the change about a minute later, it ran a new pipeline revision and passed this stage and action and deployed my changes to production.

Use CodePipeline output to modify CloudFormation template

After manually creating a pipeline, you can export its contents in JSON format using the CLI. This is helpful when you want to run it from the CLI later or through AWS CloudFormation
To pipe the contents of a specific pipeline to a JSON file, run the command below replacing  YOUR_PIPELINE_NAME with your pipeline name – as found at in your AWS CodePipeline Console.
aws codepipeline get-pipeline --name YOUR_PIPELINE_NAME > my-pipeline.json
You will see a JSON output similar to the configuration below. For more information on use this command, see get-pipeline.

 {
   "name": "cfn-nag",
   "actions": [
     {
       "inputArtifacts": [
         {
           "name": "MyApp"
         }
       ],
       "name": "cfn-nag-pipeline",
       "actionTypeId": {
         "category": "Invoke",
         "owner": "AWS",
         "version": "1",
         "provider": "Lambda"
       },
       "outputArtifacts": [],
       "configuration": {
         "FunctionName": "cfn-nag-pipeline",
         "UserParameters": "pipeline.yml"
       },
       "runOrder": 1
     }
   ]
 },

This provided a JSON structure with all the stages and actions already defined in code so I was able to copy and paste within my AWS::CodePipeline::Pipeline resources construct in the CloudFormation template. Since the CodePipeline DSL produces a slightly different case than CloudFormation, you need to update the letter case for several properties to conform to the CloudFormation standard. Moreover, if you prefer to run from YAML like me, you’ll need to convert your JSON to YAML using a tool such as JSONtoYAML.

 - Name: CfnNag
   Actions:
     - InputArtifacts:
       - Name: MyApp
       Name: cfn-nag-pipeline
       ActionTypeId:
         Category: Invoke
         Owner: AWS
         Version: '1'
         Provider: Lambda
       OutputArtifacts: []
       Configuration:
         FunctionName: cfn-nag-pipeline
         UserParameters: pipeline.yml
       RunOrder: 1

Finally, you will copy the snippet you create that looks similar to the above and incorporate it into the part of your CloudFormation template that automates the provisioning the CodePipeline resources. Once you’ve done this, you can launch a stack from this CloudFormation template so that the execution of cfn_nag is automatically run by Lambda for this pipeline.

Pricing

There are a number of services used in this solution including AWS CodePipeline, AWS Lambda, and AWS CloudWatch Logs
AWS CodePipeline
“With AWS CodePipeline, there are no upfront fees or commitments. You pay only for what you use. AWS CodePipeline costs $1 per active pipeline* per month. To encourage experimentation, pipelines are free for the first 30 days after creation.
* An active pipeline is a pipeline that has existed for more than 30 days and has at least one code change that runs through it during the month. There is no charge for pipelines that have no new code changes running through them during the month. An active pipeline is not prorated for partial months.”
More can be found at Amazon CodePipeline Pricing.
AWS Lambda
“The Lambda free tier includes 1M free requests per month and 400,000 GB-seconds of compute time per month. The memory size you choose for your Lambda functions determines how long they can run in the free tier. The Lambda free tier does not automatically expire at the end of your 12 month AWS Free Tier term, but is available to both existing and new AWS customers indefinitely.”
More can be found at Amazon Lambda Pricing.
Amazon CloudWatch Logs

• $0.50 per GB ingested**
• $0.03 per GB archived per month***

More can be found at Amazon CloudWatch Pricing.

Additional Resources

• cfn-nag-pipeline – source in GitHub
• cfn-nag-pipeline – as hosted on the AWS SAR

Stelligent Amazon Pollycast