Over the past few weeks, I’ve been describing how I’m automating the provisioning of several of the AWS Code Services including CodePipeline and Custom CodePipeline Actions. This time, I’m describing a way of provisioning AWS CodeDeploy in CloudFormation. For now, I’m doing the automation against a generic application provided by AWS. I’ll apply it to our Dromedary demo application later and publish it in a future post.
DeploymentManagement_CodeDeploy

   About AWS CodeDeploy

As AWS describes: “AWS CodeDeploy is a service that automates code deployments to any instance, including Amazon EC2 instances and instances running on-premises. AWS CodeDeploy makes it easier for you to rapidly release new features, helps you avoid downtime during application deployment, and handles the complexity of updating your applications. You can use AWS CodeDeploy to automate software deployments, eliminating the need for error-prone manual operations, and the service scales with your infrastructure so you can easily deploy to one instance or thousands.” Below, I describe the core components that make up CodeDeploy.

  • AppSpec – The appspec.yml must be in the root directory of an AWS CodeDeploy deployment bundle. Here’s an example appspec.yml file from AWS.
  • Application – The application is the container for the deployment and deployment configuration. The application uses the appspec.yml to define how the lifecycle events behave.
  • Deployment Group – Define how the deployment behaves. In the deployment group, you set the previously-defined Application, Deployment Configuration, on which EC2 instances to run the deployment and any AutoScaling configuration.
  • Deployment Configuration – There are three default configurations provided by CodeDeploy: CodeDeployDefault.OneAtATime, CodeDeployDefault.AllAtOnce and CodeDeployDefault.HalfAtATime. You can describe custom configurations as well.
  • Lifecycle Events – Each deployment triggers certain events that run in a pre-defined sequence. The actual code that is run is defined in the appspec.yml using hooks section of the file. Here are the events that get triggered as part of a typical CodeDeploy deployment.
    • ApplicationStop
    • DownloadBundle
    • BeforeInstall
    • Install
    • AfterInstall
    • ApplicationStart
    • ValidateService

In order for CodeDeploy to work, you need to install the CodeDeploy agent on each EC2 instance on which you’re running CodeDeploy.  CodeDeploy communicates with the agents via HTTPS over port 443. The agent contains code that has CodeDeploy domain-specific knowledge and uses the defined configuration to run through its lifecycle events. 

ManagementTools_CloudFormation  Executing the CloudFormation Template

Below, you see an example of running the template that I’ve defined. You’ll need to define your own stack name and EC2KeyPairName. From your AWS CLI, run a command to launch the CloudFormation stack. You can also launch the same using the CloudFormation console.

aws cloudformation create-stack --stack-name MyStackName --template-url https://s3.amazonaws.com/stelligent-training-public/public/codedeploy/codedeploy-master.json
--region us-east-1 --disable-rollback --capabilities="CAPABILITY_IAM"
--parameters ParameterKey=S3Bucket,ParameterValue=aws-codedeploy-us-east-1
ParameterKey=S3Key,ParameterValue=public/Sample_Linux_App.zip
ParameterKey=EC2TagValue,ParameterValue=CodeDeployEC2Tag
ParameterKey=EC2KeyPairName,ParameterValue=MyEC2KeyPairName

It’ll take about 10-15 minutes to launch the stacks that launch the EC2 instance, install the CodeDeploy agent, configure and run a deployment of the application. You can visually verify the application works by going to the CodeDeploy console (shown in Figure 1), select an application, then a deployment and click on the link under the Instance ID column. From the EC2 console, find the Public IP and prepend http:// to it from your web browser. codedeploy-deployment

Figure 1: AWS CodeDeploy deployment status

Architecture

There are two CloudFormation templates to define the EC2 instances, S3 Distribution Location, IAM and CodeDeploy resources along with its overall orchestration via nested stacks. They are:

  • codedeploy-master.json – The master template orchestrates the execution of the other CloudFormation templates using nested CloudFormation stacks and the DependsOn attribute. It uses the Outputs from one stack as input parameters to the next calling template. Using the DependsOn attribute ensures that the resources have been provisioned prior to calling the next template.
  • codedeploy-deployment.json – Automates the provisioning of AWS CodeDeploy resources including the CodeDeploy Application and Deployment. It points to an S3 bucket and key where the sample application zip file is stored.

The resulting infrastructure is illustrated in Figure 2. codedeploy-arch

Figure 2: Infrastructure Architecture Diagram

Implementation

There are several parts to automating the process of downloading the sample application, provisioning EC2 instances with the CodeDeploy agent, provisioning CodeDeploy to create an application and to run a deployment for that application using CodeDeploy. The first example snippet below can be found in the codedeploy-master.json CloudFormation template. This launches a new stack from the AWS CloudFormation template that provisions EC2 instances and installs the CodeDeploy agent.

    "CodeDeployEC2InstancesStack":{
      "Type":"AWS::CloudFormation::Stack",
      "Properties":{
        "TemplateURL":"http://s3.amazonaws.com/aws-codedeploy-us-east-1/templates/latest/CodeDeploy_SampleCF_Template.json",
        "TimeoutInMinutes":"60",
        "Parameters":{
          "TagValue":{
            "Ref":"EC2TagValue"
          },
          "KeyPairName":{
            "Ref":"EC2KeyPairName"
          }
        }
      }
    },

The next snippet from the same codedeploy-master.json template uses the EC2 tag value that was set in the first stack as a way for this template to determine on which EC2 instances it will run the CodeDeploy deployment. It also uses the DependsOn attribute to ensure that the stack has been created before attempting to launch this stack – since it requires resources from the first stack.

    "CodeDeploySimpleStack":{
      "Type":"AWS::CloudFormation::Stack",
      "DependsOn":"CodeDeployEC2InstancesStack",
      "Properties":{
        "TemplateURL":"https://s3.amazonaws.com/stelligent-training-public/public/codedeploy/codedeploy-deployment.json",
        "TimeoutInMinutes":"60",
        "Parameters":{
          "TagValue":{
            "Ref":"EC2TagValue"
          },
          "RoleArn":{
            "Fn::GetAtt":[
              "CodeDeployEC2InstancesStack",
              "Outputs.CodeDeployTrustRoleARN"
            ]
          },
          "Bucket":{
            "Ref":"S3Bucket"
          },
          "Key":{
            "Ref":"S3Key"
          }
        }
      }
    }

In the codedeploy-deployment.json template snippet below, I’m defining the CodeDeploy application. This generates a unique identifier that’s used in the next resource definition in the template.

    "MyApplication":{
      "Type":"AWS::CodeDeploy::Application"
    },

In the snippet below (also from codedeploy-deployment.json), I’m defining how my deployment will behave using the CodeDeploy Deployment Group. Once again, I use the DependsOn attribute to ensure that the CodeDeploy application exists prior to provisioning the deployment group since it needs to already exist. Then, I find the application bundle in S3 using the S3Location property. From the command-line execution snippet described earlier, you’ll see that I’m passing aws-codedeploy-us-east-1 as the S3 bucket parameter and samples/latest/SampleApp_Linux.zip as the S3 Key. This resolves to https://s3.amazonaws.com/aws-codedeploy-us-east-1/samples/latest/SampleApp_Linux.zip.

    "MyDeploymentGroup":{
      "Type":"AWS::CodeDeploy::DeploymentGroup",
      "DependsOn":"MyApplication",
      "Properties":{
        "ApplicationName":{
          "Ref":"MyApplication"
        },
        "Deployment":{
          "Description":"First time",
          "IgnoreApplicationStopFailures":"true",
          "Revision":{
            "RevisionType":"S3",
            "S3Location":{
              "Bucket":{
                "Ref":"Bucket"
              },
              "BundleType":"Zip",
              "Key":{
                "Ref":"Key"
              }
            }
          }
        },
        "Ec2TagFilters":[
          {
            "Key":{
              "Ref":"TagKey"
            },
            "Value":{
              "Ref":"TagValue"
            },
            "Type":"KEY_AND_VALUE"
          }
        ],
        "ServiceRoleArn":{
          "Ref":"RoleArn"
        }
      }

Troubleshooting

As you’re experimenting with CodeDeploy and automating the provisioning of CodeDeploy in CloudFormation, you’ll likely experience a few problems along the way. There were a couple of useful suggestions provided to me by AWS Support.

      • Get the logs – SSH into the EC2 instances on which the CodeDeploy agent was installed and view the contents of the /opt/codedeploy-agent/deployment-root and /var/log/ directories. You might choose to zip up the directories and download them for easy viewing as demonstrated below.

zip -r varlog.zip /var/log
zip -r deployment-root.zip /opt/codedeploy-agent/deployment-root

      • View in CloudWatch – I haven’t tried this yet, but there’s an interesting article on the AWS DevOps blog about viewing the CodeDeploy logs in CloudWatch: View AWS CodeDeploy logs in Amazon CloudWatch console
      • Turn off Auto Rollback in CloudFormation – When you’re using the AWS CloudFormation Console and you accept all the default options, it will auto rollback all the stacks. This makes it more difficult to troubleshoot when something does go wrong. Instead, you might choose to call from the command line as shown in the example below. You can still turn off auto rollback using the console, but it’s easier to forget.
aws cloudformation create-stack --stack-name MyStackName --template-body file://mystack.json --region us-east-1 --disable-rollback

Feature Backlog

There’s some additional functionality that I plan to implement in the coming weeks as described below.

      • Use Elastic Load Balancing and AutoScaling so that it’s a more highly available solution with a single endpoint
      • Create a deployment pipeline using AWS CodePipeline and automate the provisioning of the pipeline in AWS CloudFormation
      • Implement CodeDeploy in CloudFormation as part of the Dromedary Demo pipeline.
      • Include re-deployment to the same EC2 instances when an application change occurs

Useful Resources