Stelligent

Using Parameter Store with AWS CodePipeline

  • Trey McElhattan

    • Systems Manager Parameter Store is a managed service (part of AWS EC2 Systems Manager (SSM)) that provides a convenient way to efficiently and securely get and set commonly used configuration data across multiple resources in your software delivery lifecycle.
    codepipeline_ssm
    In this post, we will be focusing on the basic usage of Parameter Store and how to effectively use it as part of a continuous delivery pipeline using AWS CodePipeline. The following describes some of the capabilities of Parameter Store and the resources with which they can be used:

    At the end of this post, you will be able to launch an example solution via AWS CloudFormation.

    Working with Parameter Store

    Prerequisites

    In order to follow the examples below, you’ll need to have the AWS CLI setup on your local workstation. You can find a guide to install the AWS CLI here.

    Creating a Parameter in the Parameter Store

    To manually create a parameter in the parameter store there a few easy steps to follow:

    1. The user must sign into their AWS account and go the EC2 console.
    2. Under the Systems Manager Shared Resources section click on Parameter Store.
    3. Click Get Started Now or Create Parameter and input the following information:
      1. Name: The name that you want the parameter to be called
      2. Description(optional): A description of what the parameter does or contains
      3. Type: You can choose either a String, String List, or Secure String
    4. Click Create Parameter and it will bring you to the Parameter Store console where you can see your newly created parameter

    To create a parameter using the AWS CLI, here are examples of creating a String, SecureString, and String List:
    String:

     aws ssm put-parameter --name "HostedZoneName" --type "String" --value "stelligent.com."

    StringList: 

    aws ssm put-parameter --name "HostedZoneNames" --type "StringList" --value “stelligent.com.,google.com.,amazon.com.”

    SecureString:

     aws ssm put-parameter --name "Password" --type "SecureString" --value "Password123$"

    After running these commands, your parameter store console would look something like this:

    Getting Parameter Values using the AWS CLI

    To get a parameter String, StringList, or SecureString from the from the Parameter Store using the AWS CLI you must use the following syntax in your terminal:
    String:

    aws ssm get-parameters --names "HostedZoneName"

    The output in your terminal would look like this:

    {
    "InvalidParameters": [], 
    "Parameters": [
        {
            "Type": "String", 
            "Name": "HostedZoneName", 
            "Value": "stelligent.com."
        }
    ]
}

    And in the console:
    StringList:

     aws ssm get-parameters --names "HostedZoneNames"

    The output in your terminal would look like this:

    {
    "InvalidParameters": [], 
    "Parameters": [
        {
            "Type": "StringList", 
            "Name": "HostedZoneNames", 
            "Value": "stelligent.com.,google.com.,amazon.com."
        }
    ]
}

    And in the console:
    SecureString:

    aws ssm get-parameters --names "Password"

    The output in your terminal would look like this (the value of the parameter is encrypted):

    {
    "InvalidParameters": [], 
    "Parameters": [
        {
            "Type": "SecureString", 
            "Name": "Password", 
            "Value": "AQECAHicQXIA+CERB7LyH8+YXXUK1vqiI87oM0Wq7kgMCmGqUQAAAGowaAYJKoZIhvcNAQcGoFswWQIBADBUBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDE0kvmQLY6Ertt5BGwIBEIAnlfTl1XxzRwUzkFCBYn8P0lJ6dOdjPNQNbYgjD1+KTk/SlNJznvrF"
        }
    ]
}

    And in the console:

    Deleting a Parameter from the Parameter Store

    To delete a parameter from the Parameter Store manually, you must use the following steps:

    1. Sign into your AWS account and go the EC2 console.
    2. Under the Systems Manager Shared Resources section click on the Parameter Store tab.
    3. Select the parameter that you wish to delete
    4. Click the Actions button and select Delete Parameter from the menu

    To delete a parameter using the AWS CLI you must use the following syntax in your terminal(this works for String, StringList, and SecureString)

    aws ssm delete-parameter --name "HostedZoneName"
    aws ssm delete-parameter --name "HostedZoneNames"
    aws ssm delete-parameter --name "Password"

    Using Parameter Store in AWS CodePipeline

    Parameter Store can be very useful when constructing and running a deployment pipeline. Parameter Store can be used alongside a simple token/replace script to dynamically generate configuration files without having to manually modify those files. This is useful because you can pass through frequently used pieces of data configuration easily and efficiently as part of a continuous delivery process. An illustration of the AWS infrastructure architecture is shown below.

    In this example, we have a deployment pipeline modeled via AWS CodePipeline that consists of two stages: a Source stage and a Build stage.
    First, let’s take a look at the Source stage.

    MyCodePipeline:
    Type: AWS::CodePipeline::Pipeline
    Properties:
      ArtifactStore:
        Location: !Ref S3Bucket
        Type: S3
      RoleArn: !GetAtt [CodePipelineRole, Arn]
      Stages:
        - Name: Source
          Actions:
          - Name: GitHubSource
            ActionTypeId:
              Category: Source
              Owner: ThirdParty
              Provider: GitHub
              Version: 1
            OutputArtifacts:
              - Name: OutputArtifact
            Configuration:
              Owner: stelligent
              Repo: parameter-store-example
              Branch: master
              OAuthToken: !Ref GitHubToken

    As part of the Source stage, the pipeline will get source assets from the GitHub repository that contains the configuration file that will be modified along with a Ruby script that will get the parameter from the Parameter Store and replace the variable tokens inside of the configuration file. 
    After the Source stage completes, there’s a Build stage, where we’ll be doing all of the actual work to modify our configuration file.
    The Build stage uses the CodeBuild project (defined as the ConfigFileBuild action) to run the Ruby script that will modify the configuration file and replace the variable tokens inside of it with the requested parameters from the Parameter Store.

      ConfigFileBuild:
    Type: AWS::CodeBuild::Project
    Properties:
      Name: !Ref AWS::StackName
      Description: Changes sample configuration file
      ServiceRole: !GetAtt CodeBuildRole.Arn
      Artifacts:
        Type: CODEPIPELINE
      Environment:
        Type: LINUX_CONTAINER
        ComputeType: BUILD_GENERAL1_LARGE
        Image: aws/codebuild/eb-ruby-2.3-amazonlinux-64:2.1.6
      Source:
        Type: CODEPIPELINE
        BuildSpec: |
          version: 0.1
          phases:
            pre_build:
              commands:
                - gem install aws-sdk
            build:
              commands:
                - ruby sample_ruby_ssm.rb
          artifacts:
            files:
              - '**/*'

    The CodeBuild project contains the buildspec that actually runs the  Ruby script that will be making the configuration changes (sample_ruby_ssm.rb).

    Here is what the Ruby script looks like:

    require 'aws-sdk'

client = Aws::SSM::Client.new(region: 'us-east-1')
resp = client.get_parameters({
  names: ["HostedZoneName", "Password"], # required
  with_decryption: true,
})
hostedzonename = resp.parameters[0].value
password = resp.parameters[1].value
file_names = ['sample_ssm_config.json']
file_names.each do |file_name|
  text = File.read(file_name)

  # Display text for usability
  puts text
  # Substitute Variables
  new_contents = text.gsub(/HOSTEDZONE/, hostedzonename)
  new_contents = new_contents.gsub(/PASSWORD/, password)

  # To write changes to the file, use:
  File.open(file_name, "w") {|file| file.puts new_contents.to_s }
end

    Here is what the configuration file with the variable tokens (HOSTEDZONE, PASSWORD) looks like before it gets modified:

    {
  "Parameters" : {
    "HostedZoneName" : "HOSTEDZONE",
    "Password" : "PASSWORD"
  }
}

    Here is what the configuration file would consist of after the Ruby script pulls the requested parameters from the Parameter Store and replaces the variable tokens (HOSTEDZONE, PASSWORD). The Password parameter is being decrypted through the ruby script in this process.

    {
  "Parameters" : {
    "HostedZoneName" : "stelligent.com.",
    "Password" : "Password123$"
  }
}

    IMPORTANT NOTE:  In this example above you can see that the “Password” parameter is being returned in plain text (Password123$). The reason that is happening is because when this Ruby script runs, it is returning the secured string parameter with the decrypted value (with_decryption: true). The purpose of showing this example in this way is purely just to illustrate what returning multiple parameters into a configuration file would look like. In a real-world situation you would never want to return any password displayed in its plain text because that can present security issues and is bad practice in general. In order to return that “Password” parameter value in its encrypted form all you would simply have to do is modify the Ruby script on the 6th line and change the “with_decryption: true” to “with_decryption: false“.  Here is what the modified configuration file would look like with the “Password” parameter being returned in its encrypted form:

    Launch the Solution via CloudFormation

    To run this deployment pipeline and see Parameter Store in action, you can click the “Launch Stack” button below which will take you directly to the CloudFormation console within your AWS account and load the CloudFormation template. Walk through the CloudFormation wizard to launch the stack. 
    In order to be able to execute this pipeline you must have the following:

    aws ssm put-parameter --name "HostedZoneName" --type "String" --value "stelligent.com."
    aws ssm put-parameter --name "Password" --type "SecureString" --value "Password123$"

    As you begin to launch the pipeline in CloudFormation (Launch Stack button is located below), you will be prompted to enter this one parameter:

    Once you have passed in this initial parameter, you can begin to launch the pipeline that will make use of the Parameter Store.

    NOTE: You will be charged for your CodePipeline and CodeBuild usage.
    Once the stack is CREATE_COMPLETE, click on the Outputs tab and then the value for the CodePipelineUrl output to view the pipeline in CodePipeline.

    Additional Resources

    Here are some additional resources you might find useful:

    Summary

    In this post, you learned how to use the EC2 Systems Manager Parameter Store and some of its features. You learned how to create, delete, get, and set parameters manually as well as through the use of the AWS CLI. You also learned how to use the Parameter Store in a practical situation by incorporating it in the process of setting configuration data that is used as part of a CodePipeline continuous delivery pipeline.

    Sample Code

    The code for the examples demonstrated in this post are located at https://github.com/stelligent/parameter-store-example. Let us know if you have any comments or questions @stelligent or @TreyMcElhattan
    Stelligent is hiring! Do you enjoy working on complex problems like figuring out ways to automate all the things as part of a deployment pipeline? Do you believe in the “one-button everything” mantra? If your skills and interests lie at the intersection of DevOps automation and the AWS cloud, check out the careers page on our website.

    Stelligent Amazon Pollycast