In this two-part series, you’ll learn how to provision, configure, and orchestrate the EC2 Container Service (ECS) applications into a deployment pipeline that’s capable of deploying new infrastructure and code changes when developers commit changes to a version-control repository so that team members can release new changes to users whenever they choose to do so: Continuous Delivery.
While the primary AWS service described in this solution is ECS, I’ll also be covering the various components and services that support this solution including AWS CloudFormation, EC2 Container Registry (ECR), Docker, Identity and Access Management (IAM), VPC and Auto Scaling Services – to name a few. In part 2, I’ll be covering the integration of CodePipeline, Jenkins and CodeCommit in greater detail.
ECS allows you to run Docker containers on Amazon. The benefits of ECS and Docker include the following:
- Portability – You can build on one Linux operating system and have it work on others without modification. It’s also portable across environment types so you can build it in development and use the same image in production.
- Scalability – You can run multiple images on the same EC2 instance to scale thousands of tasks across a cluster.
- Speed – Increase your speed of development and speed of runtime execution.
“ECS is a highly scalable, high performance container management service that supports Docker containers and allows you to easily run applications on a managed cluster of Amazon EC2 instances. Amazon ECS eliminates the need for you to install, operate, and scale your own cluster management infrastructure.” [1]
The reason you might use Docker-based containers over traditional virtual machine-based application deployments is that it allows a faster, more flexible, and still very robust immutable deployment pattern in comparison with services such as traditional Elastic Beanstalk, OpsWorks, or native EC2 instances.
While you can very effectively integrate Docker into Elastic Beanstalk, ECS provides greater overall flexibility.
The reason you might use ECS or Elastic Beanstalk containers with EC2 Container Registry over similar offerings such as Docker Hub or Docker Trusted Registry is higher performance, better availability, and lower pricing. In addition, ECR utilizes other AWS services such as IAM and S3, allowing you to compose more secure or robust patterns to meet your needs.
Based on the current implementation of Lambda, the reasons you might choose to utilize ECS instead of serverless architectures include:
- Lower latency in request response time
- Flexibility in the underlying language stack to use
- Elimination of AWS Lambda service limits (requests per second, code size, total code runtime)
- Greater control of the application runtime environment
- The ability to link modules in ways not possible with Lambda functions
I’ll be using a sample PHP application provided by AWS to demonstrate Continuous Delivery pipeline using ECS, CloudFormation and, in part 2, AWS CodePipeline.
Create and Connect to a CodeCommit Repository
While you can store your application code in any version-control repository, in this example, I’ll be using the AWS CodeCommit Git repository. I’ll be integrating CodeCommit with CodePipeline. I’m basing the code from the Amazon ECS PHP Simple Demo App located at https://github.com/awslabs/ecs-demo-php-simple-app.
To create your own CodeCommit repo, follow these instructions: Create and Connect to an AWS CodeCommit Repository. Take note of the repository name as you’ll be using it as a CloudFormation user parameter in part 2. I called my CodeCommit repository ecs-demo. You can call it the same but if you do name it something different, be sure to replace the samples with your repo name.
After you create your CodeCommit repo, copy the contents from the AWS PHP ECS Demo app and commit all of the files.
CodeCommit provides the following features and benefits[2]:
- Highly available, Secure and Private Git repositories
- Use your existing Git tools
- Automatically encrypts all files in transit and at rest
- Provides Webhooks – to trigger Lambda functions or push notifications in response to events
- Integrated with other AWS services like IAM so you can define user-specific permissions
Create a Private Image Repository in ECS using ECR
A snippet of the CloudFormation template for provisioning an ECR repo is listed below.
"MyRepository":{
"Type":"AWS::ECR::Repository",
"Properties":{
"RepositoryName":{
"Ref":"AWS::StackName"
},
"RepositoryPolicyText":{
"Version":"2008-10-17",
"Statement":[
{
"Sid":"AllowPushPull",
"Effect":"Allow",
"Principal":{
"AWS":[
{
"Fn::Join":[
"",
[
"arn:aws:iam::",
{
"Ref":"AWS::AccountId"
},
":user/",
{
"Ref":"IAMUsername"
}
]
]
}
]
},
"Action":[
"ecr:GetDownloadUrlForLayer",
"ecr:BatchGetImage",
"ecr:BatchCheckLayerAvailability",
"ecr:PutImage",
"ecr:InitiateLayerUpload",
"ecr:UploadLayerPart",
"ecr:CompleteLayerUpload"
]
}
]
}
}
}
In defining an ECR, you can securely store your Docker images and refer to them when building, tagging and pushing these Docker images.
To launch the CloudFormation stack to create an ECR repository, click this button: 
Docker
“Docker containers wrap up a piece of software in a complete filesystem that contains everything it needs to run: code, runtime, system tools, system libraries – anything you can install on a server. This guarantees that it will always run the same, regardless of the environment it is running in.” [3] In this demonstration, you’ll build, tag and push a PHP application as a Docker image into an ECR repository.
Build Docker Image and Upload to ECR Locally
Prerequisites
- You’re running these commands from an Amazon Linux EC2 instance. If you’re not, you’ll need to adapt the instructions according to your OS flavor.
- You’ve created an ECR repo (see the “Create a Private Image Repository in ECS using ECR” section above)
- You’ve created a CodeCommit repository and committed the PHP code from the AWS PHP app in GitHub (see the “Create and Connect to a CodeCommit Repository” section above)
Steps
- Install Docker on an Amazon Linux EC2 instance for which your AWS CLI has been configured (you can find detailed instructions at Install Docker)
sudo yum update -y sudo yum install -y docker sudo service docker start sudo usermod -a -G docker ec2-user
- Logout and log back in and type:
docker info
- Install Git:
sudo yum -y install git*
- Clone the ECS PHP example application (if you used a different repo name, be sure to update the sample command here):
git clone ssh://git-codecommit.us-east-1.amazonaws.com/v1/repos/ecs-demo
- Change your directory:
cd ecs-demo
- Configure your AWS account by running the command below and following the prompts to enter your credentials, region and output format.
aws configure
- Run the command below to login to ECR.
eval $(aws --region us-east-1 ecr get-login)
- Build the image using Docker. Replace REPOSITORY_NAME with the ECSRepository Output from the ECR stack you launched and TAG with a unique value. Make note of the name the image tag you’re using in creating the Docker image as you’ll be using it as a input parameter to a CloudFormation stack later. If you want to use the default value, just name it latest.
docker build -t REPOSITORY_NAME:TAG .
- Tag the image (replace REPOSITORY_NAME, TAG and AWS_ACCOUNT_ID):
docker tag REPOSITORY_NAME:TAG AWS_ACCOUNT_ID.dkr.ecr.us-east-1.amazonaws.com/REPOSITORY_NAME:TAG
- Push the tagged image to ECR (replace REPOSITORY_NAME, AWS_ACCOUNT_ID and TAG):
docker push AWS_ACCOUNT_ID.dkr.ecr.us-east-1.amazonaws.com/REPOSITORY_NAME:TAG
- Verify the image was uploaded to your ECS Repository by going to your AWS ECS Console, clicking on Repositories and selecting the repository you created when you launched the ECS Stack.
Dockerfile
“A Dockerfile is a text document that contains all the commands a user could call on the command line to assemble an image. Using docker build users can create an automated build that executes several command-line instructions in succession.” [4] The snippet you see below is the Dockerfile to run the PHP sample application. You can see that it runs OS updates, installs the required packages including apache and PHP and then configures the HTTP server and port. While these are types of steps you might run in any automated build and deployment script, the difference is that it’s running these steps within a container which means that it runs very quickly, you can run these same steps across operating systems, and you can run these procedures across multiple tasks in a cluster.
FROM ubuntu:12.04 # Install dependencies RUN apt-get update -y RUN apt-get install -y git curl apache2 php5 libapache2-mod-php5 php5-mcrypt php5-mysql # Install app RUN rm -rf /var/www/* ADD src /var/www # Configure apache RUN a2enmod rewrite RUN chown -R www-data:www-data /var/www ENV APACHE_RUN_USER www-data ENV APACHE_RUN_GROUP www-data ENV APACHE_LOG_DIR /var/log/apache2 EXPOSE 80 CMD ["/usr/sbin/apache2", "-D", "FOREGROUND"]
This Dockerfile gets run when you run the docker build command. This file has been committed to my CodeCommit repo as you can see in the figure below.
AWS CodeCommit repository for a PHP application illustrating Dockerfile location
Create an ECS Environment in CloudFormation
In this section, I’m describing the how to configure the entire ECS stack in CloudFormation. This includes the architecture, its dependencies, and the key CloudFormation resources that make up the stack.
Architecture
The overall solution architecture is illustrated in the CloudFormation diagram below.
Provisioning, Configuring and Orchestrating an EC2 Container Service Architecture
- Auto Scaling Group – I’m using an auto scaling group to scale the underlying EC2 infrastructure in the ECS Cluster. It’s used in conjunction with the Launch Configuration.
- Auto Scaling Launch Configuration – I’m using a launch configuration to scale the underlying EC2 infrastructure in the ECS Cluster. It’s used in conjunction with the Auto Scaling Group.
- CodeCommit – I’m using CodeCommit as my Git repo to store the application and infrastructure code.
- CodePipeline – CodePipeline describes my Continuous Delivery workflow. In particular, it integrates with CodeCommit and Jenkins to run actions every time someone commits new code to the CodeCommit repo. This will be covered in more detail in part 2.
- ECS Cluster – “An ECS cluster is a logical grouping of container instances that you can place tasks on.”[6]
- ECS Service – With an ECS service, you can run a specific number of instances of a task definition simultaneously in an ECS cluster [5]
- ECS Task Definition – A task definition is the core resource within ECS. This is where you define which Docker images to run, CPU/Memory, ports, commands and so on. Everything else in ECS is based upon the task definition
- Elastic Load Balancer – The ELB provides the endpoint for the application. The ELB dynamically determines which EC2 instance in the cluster is serving the running ECS tasks at any given time.
- IAM Instance Profile – “An instance profile is a container for an IAM role that you can use to pass role information to an EC2 instance when the instance starts.” [7] In the sample, I’m using the instance profile to define the roles for which launch configurations use as part of the underlying EC2 instance that the ECS cluster runs
- IAM Roles – I’m describing roles that have access to certain AWS resources for the EC2 instances (for ECS), Jenkins and CodePipeline
- Jenkins – I’m using Jenkins to execute the actions that I’ve defined in CodePipeline. For example, I have a bash script that updates the CloudFormation stack when an ECS Service is update. This action is orchestrated via CodePipeline and then executed on te Jenkins server on one of its configured jobs. This will be covered in more detail in part 2.
- Virtual Private Cloud (VPC) – In the CloudFormation template, I’m using a VPC template that we developed to define VPC resources such as: VPCGatewayAttachment, SecurityGroup, SecurityGroupIngress, SecurityGroupEgress, SubnetNetworkAclAssociation, NetworkAclEntry, NetworkAcl, SubnetRouteTableAssociation, Route, RouteTable, InternetGateway, and Subnet
Dependencies
There are four core dependencies in this solution: EC2 Key Pair, CodeCommit Repo, a VPC, and an ECR repo and Docker Image
- EC2 Key Pair – A key pair for which you have access. See Create a Key Pair.
- CodeCommit – In this demo, I’m using an AWS CodeCommit Git repo to store the PHP application code along with my Docker configuration. See the instructions for configuring a Git repo in CodeCommit above
- VPC – This template requires an existing AWS Virtual Private Cloud has been created
- ECR repo and image – You should have created an E2 Container Service Repository (ECR) using the CloudFormation template from the previous section. You should have also built, tagged and pushed a Docker image to ECR using the instructions described at Create a Private Image Repository in ECS using ECR above
ECS Cluster
With an ECS Cluster, you can manage multiple services. An ECS Container Instance runs an ECS agent that is registered to the ECS Cluster. To define an ECS Cluster in CloudFormation, use the Cluster resource: AWS::ECS::Cluster as shown below.
"EcsCluster":{
"Type":"AWS::ECS::Cluster",
"DependsOn":[
"MyVPC"
]
},
ECS Service
An ECS Service defines a task definition and a desired number of task instances. A service manages tasks of a specified task definition.
In the context of ECS, an ELB distributes load between the different EC2 instances hosting your tasks, so you can optionally create a new ELB when creating a service.
To define an ECS Service in CloudFormation, use the Service resource: AWS::ECS::Service.
"EcsService":{
"Type":"AWS::ECS::Service",
"DependsOn":[
"MyVPC",
"ECSAutoScalingGroup"
],
"Properties":{
"Cluster":{
"Ref":"EcsCluster"
},
"DesiredCount":"1",
"DeploymentConfiguration":{
"MaximumPercent":100,
"MinimumHealthyPercent":0
},
"LoadBalancers":[
{
"ContainerName":"php-simple-app",
"ContainerPort":"80",
"LoadBalancerName":{
"Ref":"EcsElb"
}
}
],
"Role":{
"Ref":"EcsServiceRole"
},
"TaskDefinition":{
"Ref":"PhpTaskDefinition"
}
}
},
Notice that I defined a DeploymentConfiguration with a MinimumHealthyPercent of 0. Since I’m only using one EC2 instance in development, the ECS service would fail during a CloudFormation update so by setting the MinimumHealthyPercent to zero, the application will experience a few seconds of downtime during the update. Like most applications/services these days, if I need to continual uptime, I’d increase the number of instances in my Auto Scaling Group and increase the MinimumHealthyPercent property.
Task Definition
With an ECS Task Definition, you can define multiple Container Definitions and volumes. With a Container Definition, you define port mappings, environment variables, CPU Units and Memory. An ECS Volume is a persistent volume to mount and map to container volumes.
To define an ECS Task Definition, use the ECS Task Definition resource: AWS::ECS::TaskDefinition.
"PhpTaskDefinition":{
"Type":"AWS::ECS::TaskDefinition",
"DependsOn":[
"MyVPC"
],
"Properties":{
"ContainerDefinitions":[
{
"Name":"php-simple-app",
"Cpu":"10",
"Essential":"true",
"Image":{
"Fn::Join":[
"",
[
{
"Ref":"AWS::AccountId"
},
".dkr.ecr.us-east-1.amazonaws.com/",
{
"Ref":"ECSRepoName"
},
":",
{
"Ref":"ImageTag"
}
]
]
},
"Memory":"300",
"PortMappings":[
{
"HostPort":80,
"ContainerPort":80
}
]
}
],
"Volumes":[
{
"Name":"my-vol"
}
]
}
},
Auto Scaling
To define an Auto Scaling Group, use the Auto Scaling Group resource in CloudFormation: AWS::AutoScaling::AutoScalingGroup.
"ECSAutoScalingGroup":{
"Type":"AWS::AutoScaling::AutoScalingGroup",
"DependsOn":[
"MyVPC"
],
"Properties":{
"VPCZoneIdentifier":[
{
"Ref":"publicSubnet01"
},
{
"Ref":"publicSubnet02"
}
],
"LaunchConfigurationName":{
"Ref":"ContainerInstances"
},
"MinSize":"1",
"MaxSize":{
"Ref":"MaxSize"
},
"DesiredCapacity":{
"Ref":"DesiredCapacity"
}
},
"CreationPolicy":{
"ResourceSignal":{
"Timeout":"PT15M"
}
},
"UpdatePolicy":{
"AutoScalingRollingUpdate":{
"MinInstancesInService":"1",
"MaxBatchSize":"1",
"PauseTime":"PT15M",
"WaitOnResourceSignals":"true"
}
}
},
To define a Launch Configuration, use the Launch Configuration resource in CloudFormation: AWS::AutoScaling::LaunchConfiguration.
"ContainerInstances":{
"Type":"AWS::AutoScaling::LaunchConfiguration",
"DependsOn":[
"MyVPC"
],
"Metadata":{
"AWS::CloudFormation::Init":{
"config":{
"commands":{
"01_add_instance_to_cluster":{
"command":{
"Fn::Join":[
"",
[
"#!/bin/bash\n",
"echo ECS_CLUSTER=",
{
"Ref":"EcsCluster"
},
" >> /etc/ecs/ecs.config"
]
]
}
}
},
"files":{
"/etc/cfn/cfn-hup.conf":{
"content":{
"Fn::Join":[
"",
[
"[main]\n",
"stack=",
{
"Ref":"AWS::StackId"
},
"\n",
"region=",
{
"Ref":"AWS::Region"
},
"\n"
]
]
},
"mode":"000400",
"owner":"root",
"group":"root"
},
"/etc/cfn/hooks.d/cfn-auto-reloader.conf":{
"content":{
"Fn::Join":[
"",
[
"[cfn-auto-reloader-hook]\n",
"triggers=post.update\n",
"path=Resources.ContainerInstances.Metadata.AWS::CloudFormation::Init\n",
"action=/opt/aws/bin/cfn-init -v ",
" --stack ",
{
"Ref":"AWS::StackName"
},
" --resource ContainerInstances ",
" --region ",
{
"Ref":"AWS::Region"
},
"\n",
"runas=root\n"
]
]
}
}
},
>
IAM
To define an IAM Instance Profile, use the InstanceProfile resource in CloudFormation: AWS::IAM::InstanceProfile.
"EC2InstanceProfile":{
"Type":"AWS::IAM::InstanceProfile",
"DependsOn":[
"MyVPC"
],
"Properties":{
"Path":"/",
"Roles":[
{
"Ref":"EC2Role"
}
]
}
},
"JenkinsRole":{
"Type":"AWS::IAM::Role",
"Properties":{
"AssumeRolePolicyDocument":{
"Statement":[
{
"Sid":"",
"Effect":"Allow",
"Principal":{
"Service":"ec2.amazonaws.com"
},
"Action":"sts:AssumeRole"
}
]
},
"Path":"/"
}
},
To define an IAM Role, use the IAM Role resource in CloudFormation: AWS::IAM::Role. The snippet below is for the EC2 role.
"EC2Role":{
"Type":"AWS::IAM::Role",
"DependsOn":[
"MyVPC"
],
"Properties":{
"AssumeRolePolicyDocument":{
"Statement":[
{
"Effect":"Allow",
"Principal":{
"Service":[
"ec2.amazonaws.com"
]
},
"Action":[
"sts:AssumeRole"
]
}
]
},
"Path":"/",
"Policies":[
{
"PolicyName":"ecs-service",
"PolicyDocument":{
"Statement":[
{
"Effect":"Allow",
"Action":[
"ecs:CreateCluster",
"ecs:RegisterContainerInstance",
"ecs:DeregisterContainerInstance",
"ecs:DiscoverPollEndpoint",
"ecs:Submit*",
"ecr:*",
"ecs:Poll"
],
"Resource":"*"
}
]
}
}
]
}
},
The snippet below is for defining the ECS IAM role.
"EcsServiceRole":{
"Type":"AWS::IAM::Role",
"Properties":{
"AssumeRolePolicyDocument":{
"Statement":[
{
"Effect":"Allow",
"Principal":{
"Service":[
"ecs.amazonaws.com"
]
},
"Action":[
"sts:AssumeRole"
]
}
]
},
"Path":"/",
"Policies":[
{
"PolicyName":"ecs-service",
"PolicyDocument":{
"Statement":[
{
"Effect":"Allow",
"Action":[
"elasticloadbalancing:Describe*",
"elasticloadbalancing:DeregisterInstancesFromLoadBalancer",
"elasticloadbalancing:RegisterInstancesWithLoadBalancer",
"ec2:Describe*",
"ec2:AuthorizeSecurityGroupIngress"
],
"Resource":"*"
}
]
}
}
]
}
},
EC2
To define security group ingress within a VPC, use the SecurityGroupIngress resource in CloudFormation: AWS::EC2::SecurityGroupIngress.
"InboundRule":{
"Type":"AWS::EC2::SecurityGroupIngress",
"DependsOn":[
"MyVPC"
],
"Properties":{
"IpProtocol":"tcp",
"FromPort":"0",
"ToPort":"65535",
"SourceSecurityGroupId":{
"Fn::GetAtt":[
"TargetSG",
"GroupId"
]
},
"GroupId":{
"Fn::GetAtt":[
"TargetSG",
"GroupId"
]
}
}
},
To define the security group egress within a VPC, use theSecurityGroupEgress resource in CloudFormation: AWS::EC2::SecurityGroupEgress.
"OutboundRule":{
"Type":"AWS::EC2::SecurityGroupEgress",
"DependsOn":[
"MyVPC"
],
"Properties":{
"IpProtocol":"tcp",
"FromPort":"0",
"ToPort":"65535",
"DestinationSecurityGroupId":{
"Fn::GetAtt":[
"TargetSG",
"GroupId"
]
},
"GroupId":{
"Fn::GetAtt":[
"SourceSG",
"GroupId"
]
}
}
},
To define the security group within a VPC, use the SecurityGroup resource in CloudFormation: AWS::EC2::SecurityGroup.
"SourceSG":{
"Type":"AWS::EC2::SecurityGroup",
"DependsOn":[
"MyVPC"
],
"Properties":{
"VpcId":{
"Ref":"MyVPC"
},
"GroupDescription":"Sample source security group",
"SecurityGroupIngress":[
{
"IpProtocol":"tcp",
"FromPort":"80",
"ToPort":"80",
"CidrIp":"0.0.0.0/0"
}
],
"Tags":[
{
"Key":"Name",
"Value":{
"Fn::Join":[
"",
[
{
"Ref":"AWS::StackName"
},
"-SourceSG"
]
]
}
}
]
}
},
ELB
To define the ELB, use the LoadBalancer resource in CloudFormation: AWS::ElasticLoadBalancing::LoadBalancer.
"EcsElb":{
"Type":"AWS::ElasticLoadBalancing::LoadBalancer",
"DependsOn":[
"MyVPC"
],
"Properties":{
"Subnets":[
{
"Ref":"publicSubnet01"
},
{
"Ref":"publicSubnet02"
}
],
"Listeners":[
{
"LoadBalancerPort":"80",
"InstancePort":"80",
"Protocol":"HTTP"
}
],
"SecurityGroups":[
{
"Ref":"SourceSG"
},
{
"Ref":"TargetSG"
}
],
"HealthCheck":{
"Target":"HTTP:80/",
"HealthyThreshold":"2",
"UnhealthyThreshold":"10",
"Interval":"30",
"Timeout":"5"
}
}
},
Summary
In this first part of the series, you learned how to use CloudFormation to fully automate the provisioning of the EC2 Container Service and Docker which includes ELB, Auto Scaling, and VPC resources. You also learned how to setup a CodeCommit repository.
In the next and last part of this series, you’ll learn how to orchestrate all of the changes into a deployment pipeline to achieve Continuous Delivery using CodePipeline and Jenkins so that any change made to the CodeCommit repo can be deployed to production in an automated fashion. I’ll provide access to all the code resources in part 2 of this series. Let us know if you have any comments or questions @stelligent or @paulduvall.
Stelligent is hiring! Do you enjoy working on complex problems like figuring out ways to automate all the things as part of a deployment pipeline? Do you believe in the “everything-as-code” mantra? If your skills and interests lie at the intersection of DevOps automation and the AWS cloud, check out the careers page on our website.
Resources
Here’s a list of some of the resources described in this post:
- Getting Started with Amazon ECS
- Amazon EC2 Container Service Template Snippets – These are snippets on which I based my ECS CloudFormation solution
- EC2 Security Group and Ingress Rule
- Docker Basics – Instructions for installing and running Docker on Amazon Linux – including uploading to the ECS Repository (ECR)
- Amazon ECS PHP Simple Demo App – This is the demo code that I used from AWS to upload to my CodeCommit repo
- ECS Repositories in US-EAST-1
- Deleting an ECS Image in ECR
- EC2 Container Service Update – Container Registry, ECS CLI, AZ-Aware Scheduling, and More
- Basic Git Commands – Pretty much all you’ll need to know to do about 80% of the commands you’ll be regularly using
- Introduction to AWS CodeCommit – Short video on using the CodeCommit git repository and its benefits
- CloudFormation resources covered in this post
- AWS::AutoScaling::LaunchConfiguration
- AWS::IAM::InstanceProfile
- AWS::IAM::Role
- AWS::AutoScaling::AutoScalingGroup
- AWS::ECS::Cluster
- AWS::ElasticLoadBalancing::LoadBalancer
- AWS::EC2::SecurityGroupIngress
- AWS::EC2::SecurityGroupEgress
- AWS::EC2::SecurityGroup (Source, Target)
- AWS::ECS::Service
- AWS::ECS::TaskDefinition
- AWS::ECR::Repository
Acknowledgements
My colleague Jeff Bachtel provided the thoughts on reasons why some teams might choose to use Docker and ECS over serverless. I also used several resources from AWS including the PHP sample app, the Introduction to AWS CodeCommit video, the CodePipeline Starter Kit and the ECS CloudFormation snippets.
|
Stelligent Amazon Pollycast
 |
