Since our entire focus at Stelligent is to help our customers apply Continuous Delivery in Amazon Web Services (AWS), we were really excited when we learned that AWS CloudFormation added support for AWS CodePipeline. I spent some time upgrading our Dromedary demo scripts to incorporate this new functionality. This article describes the process we applied in making the necessary changes along with some bonus functionality we were able to implement a more fully automated solution – thanks to Corey at AWS Support.
When the AWS CodePipeline team released its Continuous Delivery service in July 2015, they provided a custom JSON-based DSL for it which we used to automate the provisioning of CodePipeline itself but we didn’t get it into the 100% automated state for which we’d been hoping. It was close, but were “chomping at the bit” for the time when AWS would begin providing CloudFormation support. Thankfully, now’s the time.
A couple of Stelligent’s engineers wrote the first version of the demo that we used at the “Infrastructure as Code” breakout session as part of AWS re:Invent 2015 so they’d already been through some of the things I learned while adding and updating our scripts.
Below, you’ll find the steps for getting this demo running on your own AWS account. All of the code is freely available and open source.

Running from CloudFormation

Click on the Launch Stack button below.


Enter the required parameters. In particular: Stack name, DDBTableName, GitHubToken, KeyName and ProdHostedZone. See the figure below and complete the rest of the steps to launch the stack. It’ll take approximately 20 minutes to launch the stack and then launch a CodePipeline pipeline instance that launches an application stack. You can see the README for other ways to run this template from CloudFormation.
dromedary_master
Once all the required AWS resources are bootstrapped, it automatically launches a pipeline instance in CodePipeline as shown below.

codepipeline_cfn

Running from the Command Line

You’ll need to run the command below from a computer on which the AWS CLI has been installed.
Launching CloudFormation stack from command line

aws cloudformation create-stack
--stack-name DromedaryStack
--template-body https://raw.githubusercontent.com/stelligent/dromedary/master/pipeline/cfn/dromedary-master.json
--region us-east-1
--disable-rollback --capabilities="CAPABILITY_IAM"
--parameters ParameterKey=KeyName,ParameterValue=YOURKEYPAIR
    ParameterKey=Branch,ParameterValue=master
    ParameterKey=BaseTemplateURL,ParameterValue=https://s3.amazonaws.com/stelligent-public/cloudformation-templates/github/dromedary/
    ParameterKey=GitHubUser,ParameterValue=YOURGITHUBUSER
    ParameterKey=GitHubToken,ParameterValue=YOURGITHUBTOKEN
    ParameterKey=DDBTableName,ParameterValue=YOURUNIQUEDDBTABLENAME
    ParameterKey=ProdHostedZone,ParameterValue=.YOURHOSTEDZONE

For more information, you can see the README at https://github.com/stelligent/dromedary.
The steps I went through to create this CloudFormation template were fairly straightforward. First, I started with a CloudFormation template that I had implemented for another effort and removed most everything except for the core structure including the AWSTemplateFormatVersion, an empty Parameters block and an empty Resources block. The core Resources Type for CodePipeline is – as you might’ve guessed – AWS::CodePipeline::Pipeline. After this, I got the name of an existing pipeline that we’d created using the AWS CodePipeline JSON DSL from our AWS CodePipeline and ran this command:
aws codepipeline get-pipeline --name YOUR_PIPELINE_NAME > pipeline.json

This provided a JSON structure with all the stages and actions already defined in code so I was able to copy and paste within my AWS::CodePipeline::Pipeline Resources construct in the CloudFormation template. Since the CodePipeline DSL produces a slightly different case than CloudFormation, I needed to update the letter case for several properties to conform to the CloudFormation standard.
Then, I added several parameters to the template including GitHubToken and GitHubUser. Finally, I ran through several testing scenarios. Be sure to run through all the steps in README as well since it’s important that you enable security for Jenkins.

Summary

Essentially, until AWS implements a service in CloudFormation, we don’t really consider it a service we can use in the manner we like so we’re really happy that CodePipeline can now be provisioned in CloudFormation. Moreover, when implementing the CodePipeline provisioning in CloudFormation, we configured it so that all of CodePipeline including the Github configuration is automated so there are no manual steps anymore (except for enabling Jenkins Global Security). If you have any questions or comments, you can contact us at info@stelligent.com or consult the contact information at https://github.com/stelligent/dromedary.
Stelligent is an expert in deploying continuous delivery pipelines in AWS. If you are looking for help moving your applications into CodePipeline, or another type of continuous delivery pipeline, we can definitely help you with that. Reach out to us here.

Useful Resources