Stelligent

Implementing Amazon Inspector in Automated AMI Pipelines

  • Xavier Williams

    • This is part two of a two part series on Amazon Inspector. This article is about implementing Amazon Inspector in automated AMI pipelines. The first article is an Introduction to Amazon Inspector.

    Automated security scanning is an essential part of DevSecOps, however setting the scanning up can be cumbersome. This quickstart incorporates Amazon Inspector and helps obviate these issues:

    Integrating Amazon Inspector into the image creation process removes many of the pain points from the list presented above. This article will go over using Inspector scanning in an AMI creation pipeline. The pipeline will only use these rules packages with Inspector:

    AWS Services used in this quickstart

    Below is a list of the services used in this implementation:

     

    Pipeline Architecture Diagram

    The CloudFormation template aws-inspector-pipeline.yaml deploys all the resources after the required parameters are filled in. 

     

    Launching the Pipeline CloudFormation Template

    Launching the pipeline requires a few prerequisite steps but after the steps are completed everything will stand up with CloudFormation in a one-click deployment style. The template can be launched from the CloudFormation template or with the AWS CLI.

    Required Prerequisites

    Forking Repository

    You will need to create a fork of the repository below:

    Creating Github Access Token

    Creating EC2 keypair

    Repository Parameters

    All parameters are located in the `aws-inspector-pipeline.yaml`

    Once all these steps have been completed, the CloudFormation stack can be launched.

    Optional Prerequisites

    The desired Inspector scan duration can be set by updating the value for the parameter `ScanLength` in the `aws-inspector-pipeline.yaml`. The unit of time is in seconds. The default value is set to 3 minutes.

    Verifying Deployment

    Once the CloudFormation template has completed launching, a 4 stage pipeline will be in CodePipeline named `<Your CloudFormation Stack Name>-codepipeline`.

    There should also be a Lambda function named “<Your CloudFormation Stack Name>-InspectorLambda*“ in the Lambda console.

    An acceptance test has been included in the repository to verify the resource creations. The acceptance test is named `verify_resources.py`.The test will call the AWS API using boto3 and print the CodePipeline name, Lambda function name, S3 reports bucket name, and the ARN of the SNS topic used to notify the user when new scans complete. The output should look like the output shown below.

    See the gist on github.

    Steps to run `verify_resources.py`

    1. Configure a default AWS profile or the AWS environment variables on the machine intended to run the script
    2. Have Python 3 installed on the machine intended to run the script
    3. Install the dependencies listed in the `requirements.txt` file from the repository
    4. Provide the name of the Inspector Pipeline CloudFormation stack as a command line argument

    Breaking Down Pipeline Stages

    The full pipeline consists of four stages:

    1. Checkout from source
    2. Build AMI
    3. Deploy AMI and Inspector resources
    4. Initiate Inspector scan on the deployed instance 

    When an Inspector scan is complete, the Lambda function is triggered via SNS. On execution the function puts the report in S3 and cleans up the remaining resources launched from the pipeline. 

    BuildAMI Stage

    The build stage of the pipeline starts by installing Packer on the CodeBuild instance, then it builds the AMI based on the packer template ami.json. 

    The ami.json file is:

    See the gist on github.

    After the image build finishes, it is tagged with the CommitID for later use in the pipeline.

    DeployAMI Stage

    The deploy stage of the pipeline pulls the AMI from the previous stage by using the AWS CLI to query the AMI using the commitID tag placed on the image in the Build stage.

    See the gist on github.

    Once the AMI ID has been pulled, the AMI is deployed along with the respective Inspector resources for the scan. The deploy stage uses the following CloudFormation template. The CloudFormation stack is launched with the name of the stack used to stand up the CodePipeline instance with the commit ID appended to it. The stack is named this way so it can be referenced based on the commit ID in later parts of the pipeline.

    See the gist on github.

    The CloudFormation template takes in the AMI ID and commit ID and adds them as UserAttributesForFindings for the Inspector assessment template resource. The stack name resulting from this template is also added to the UserAttributesForFindings of the Inspector assessment template.

    Each Inspector resource ARN is placed in the stack outputs of this template.

    ScanAMI Stage

    The scan stage of the pipeline the Inspector scan is initiated using the boto3 Python code in start_scan.py. The methods for the class used in this Python file are located in InspectorQuickstart.py. Within the InspectorQuickstart class, the stack name of the CloudFormation launched in the previous template is used to look up the stack outputs so a run can be initiated based on the assessment template from the stack. The assessment template also subscribes to the SNS template launched in the CodePipeline template so it can send SNS messages to the Lambda function on completion.

    Lambda Function

    When the Lambda function is triggered via SNS, it parses the message to find the name of the completed assessment run. Once the lambda has the ARN of the assessment run, the results are uploaded to S3, and the function sends a SNS message to the ScanCompleteTopic notifying anyone subscribed with the AMI ID used in the scan and the S3 location of the full report. The Lambda function also deletes the CloudFormation stack deployed in the DeployAMI stage. This function pulls information about the Inspector deployment by querying the UserAttributesForFindings placed on the Inspector assessment template in the DeployAMI stage. The Lambda function code is posted below:

    See the gist on github.

    Viewing Scan Results

    In order to view the scan results, you’ll need to navigate to the S3 bucket where they are stored. After standing up a CloudFormation stack from the `aws-inspector-pipeline.yaml`, you’ll need to navigate to the outputs section of the CloudFormation stack and look for the value of the output `ReportsBucket` to see what the name of the S3 bucket the stack created. Navigate to the bucket to see your reports. The reports are stored in a folder structure `date of scan/commit id/`. If you want to receive notifications whenever scans are completed, you can subscribe to the SNS topic that is displayed in the stack output `NotificationSNS`. Be sure to confirm the subscription after subscribing. The SNS notification will contain the AMI ID that the instance being scanned was launched from and the S3 URI of the results. 

    Switching the Packer Script

    The default configuration of this quickstart builds the AMI out of a Packer script that does not address any of the findings found in the report. A second Packer script is included in the repository named `hardened_ami.json` that includes some lockdowns in the image creation process. This Packer script can be used instead of the `ami.json` script by replacing the parameter value `PackerFile` in the `aws-inspector-pipeline.yaml` file. After making this change, launch the pipeline and inspect the findings results. The findings count should be lower than the scan results of AMI’s created with the previous Packer script.

    Here’s the snippet of the Packer script hardened_ami.json that addresses some of the findings:

    See the gist on github.

    Next Steps

    This demo is a proof of concept for using Inspector in an AMI pipeline. This pipeline could be extended to include other steps or security checks depending on the use case. The ami.json file could become more customized based on the application meant to run on it. Security hardening could be added into the AMI configuration, with the intent to add more lockdowns whenever new findings come in. This demo also provides a seamless way to start using Inspector without worry of unknown configurations issues up front. If you need to learn how to interpret Inspector scan results, our article Introduction to Amazon Inspector goes over scan results.

    Use this quickstart if you’re trying to start creating an AMI pipeline. If you already have an AMI Pipeline, use the lessons in this quickstart to add automated instance scanning to your pipeline. You should extend or modify pieces of this quickstart to fit your specific use-cases. This quickstart gives you everything you need to get up and running with Amazon Inspector. If you decide to try the quickstart, please reach out to us and let us know how it goes!

    Featured image by Craig Whitehead on Unsplash

    Stelligent Amazon Pollycast