When using serverless on AWS, you do not need to worry about load balancing, auto scaling, operating system management, managing utilization, or underlying hardware failures. All of it is abstracted from you so that you can focus on coding. What’s more, since you only pay for what you use, you can do more experimentation. It also lends itself to writing smaller, more purpose-built capabilities for end users.

Continuous Delivery and Continuous Deployment embrace DevOps principles of increasing the frequency of useful feedback between builders and end users; this helps builders create better products.

In this post, I describe a proof of concept solution that you can launch in your AWS account for running Continuous Deployment of a serverless application. The solution uses 10 fully-managed services on AWS and deploys every source commit to “production” – if all stages succeed. This approach accelerates those rapid feedback loops between end users and builders.

At this point, the serverless application simply stores data and retrieves it through an API. While it is simple, it exercises many of the services in a typical serverless application that can handle millions of end users.

The application gets data from an Amazon DynamoDB table via Amazon API Gateway and AWS Lambda. You can see this data by accessing a URL generated by API Gateway. See this video to walkthrough the steps in launching this solution.

Figure 1 shows the deployment pipeline provisioned in this solution.

CodePipeline

Figure 1 – Deployment Pipeline in AWS CodePipeline for the Serverless Web Application

The stages and actions are described in more details below.

Source

  • The Source action in the Source stage pulls source files from a GitHub repository. This repository is populated from an S3 bucket that is provided as a parameter when launching the CloudFormation stack.

Build

  • The BuildSAM action in this stage runs a CodeBuild build provider to package the serverless application source files used by the AWS Serverless Application Model (SAM) and stores the resulting zip file in a CodeBuild artifact that is used by CodePipeline.

Deploy

  • The GenerateChangeSet action in this stage uses a CloudFormation deploy provider to generate a CloudFormation change set based on the CodePipeline artifacts generated by the BuildSAM action.
  • The ExecuteChangeSet action in this stage uses a CloudFormation deploy provider to execute a CloudFormation change set which deploys the resources defined in the SAM template.
  • The InsertData action in this stage uses a CodeBuild test provider to run a command that inserts data into a DynamoDB table.

Once the pipeline is complete, you can click on a CloudFormation Output value generated by the SAM stack. Figure 2 shows the data returned when accessing the URL generated by API Gateway.

Figure 2 – Data displayed when entering URL that accesses API generated by API Gateway

Continuous Delivery

Continuous Delivery is the full automation of your release process in which your software is always in a releasable state. With every code change, a deployment pipeline builds, tests, deploys, and makes the software available to release to users. If anything fails, the deployment pipeline will stop and notify builders so that they can quickly commit changes to source control to fix the problem.

The major benefit of Continuous Delivery is that you can get useful and frequent feedback on which to build better software for users.

Continuous Delivery is part of set of practices for modern application development. Figure 3 shows the difference between some of these modern practices such as continuous integration, continuous delivery, and continuous deployment.

CI and CD

Figure 3 – Continuous practices of modern application development [Source] 

Continuous Integration focuses on getting extremely quick and useful feedback to builders by running a build, static analysis, and unit tests in a development environment with every code change committed to source control.

Continuous Delivery assumes that good Continuous Integration practices are present and extends the pipeline to deploying changes to environments that are more like production. Longer-running tests are executed in these environments and the environments themselves are provisioned and configured based on a fully automated process. The very last part of a Continuous Delivery process is a human decision to release the software by clicking a button. This might include various approvals in a fully automated workflow.

Continuous Deployment extends on Continuous Delivery. The critical difference is that – with Continuous Deployment – changes are automatically deployed to production environments if all of the builds, tests, deployments, and other checks pass.

Serverless

Here are some of key aspects of serverless on AWS.

  • No infrastructure provisioning or management – The infrastructure provider (i.e. AWS) manages the technical operations such as monitoring, scaling, and security.
  • Automatic scaling – Serverless services can scale based on demand without any additional changes.
  • Pay for value – Pay only for what you use (i.e. do not pay for “idle”). This often leads to lower costs
  • Highly available and secure – The services build in the redundancies so that builders (and end users) are less likely to experience downtime. Things like automatic patching occur without any changes or notice on behalf of builders.

All of this means that – as a builder – you are able to rapidly provide more value to customers. This is because you are spending less time on infrastructure and more time on coding and experimenting for end users.

Automated Solution Instructions

With this automated solution, you will run a single command to launch a CloudFormation Stack that provisions all of the resources in this solution. Then, you will go to your web browser to see the data returned from an Amazon DynamoDB table via Amazon API Gateway and AWS Lambda.

Prerequisites

  • AWS Account and permissions for the required AWS services.
  • AWS CLI installed and access keys configured.
  • While most IDEs should work, only AWS Cloud9 is supported for this solution. Git is automatically installed with Cloud9.

Launch CloudFormation Stack

To reduce variation, you can run the commands from an AWS Cloud9 IDE.  Regardless, you will need to ensure your environment has access credentials for running the AWS CLI.

Before running the commands to launch the stack, you should create a secret in AWS Secrets Manager by running the command below – replacing GITHUBTOKEN with your GitHub token you created at https://github.com/settings/tokens.

aws secretsmanager create-secret --name github/personal-access-token --description "GitHub Token" --secret-string "GITHUBTOKEN"

To launch the solution, run the commands from Listing 1. The bash script uses the AWS CLI to create a CloudFormation stack that provisions S3, IAM, CodeBuild, and CodePipeline resources.

It takes about 3 minutes to launch the stack and run through the pipeline.

Listing 1 – Commands to Launch Serverless Web Apps CD Solution

It will take a few minutes to initiate the launch of the stack and then another five minutes to launch all the resources. Once the cloudproviders-pmd-us-east-1 stack is complete, continue to the next step to get the data.

Get Data from API through Browser

  1. Once the cloudproviders-pmd-us-east-1 CloudFormation stack is complete, click on the Outputs pane.
  2. Click on the URL in the Value column for the GetDataApi Key.
  3. Three rows of data are returned. This data is pulled from the DynamoDB table via AWS API Gateway and AWS Lambda.

Architecture and Workflow

As a user, when you click on the URL provided as a CloudFormation Stack Output that is generated by API Gateway, it accesses API Gateway which is configured to trigger an AWS Lambda Function. This function retrieves the data from a DynamoDB table and sends the response back through API Gateway so that users get a list of data stored in the DynamoDB table.

Figure 4 shows the workflow between components beginning with the triggering event which is a user accessing the URL from a web browser.

Figure 4 – User workflow of the serverless web application solution

As an engineer, you will run a simple AWS CLI command to launch the CloudFormation Stack from the CloudFormation template. When launched, this stack automatically provisions resources for storing files, storing code, running builds, and orchestrating the deployment pipeline of the key solution resources.

The deployment pipeline automatically provisioned by the CloudFormation stack is defined in AWS CodePipeline. This pipeline uses a GitHub repository for storing code, a couple of CodeBuild projects for building AWS Lambda functions, and CloudFormation deploy providers that deploy the serverless resources using an AWS SAM template.

The AWS SAM template is an extension to CloudFormation that makes it easier to build serverless applications using a simple and clean syntax (i.e. much less configuration code). In this solution, the SAM template defines the API Gateway, Lambda, and DynamoDB resources that get provisioned.

All of the resources in this solution are defined in greater detail below.

Figure 5 shows the groupings and components of this serverless web application.

Figure 5 – Serverless Web Application Architecture

CloudFormation Stack

  • Amazon S3 – Store files in object storage. In this solution, I am using S3 to store the artifacts used by CodePipeline, and store the files used by the static website. The CloudFormation template defines this using the AWS::S3::Bucket resource.
  • AWS IAM – Service for defining permissions to AWS resources. In this solution, I am using IAM to define roles for access to CodePipeline, CodeBuild, and the CloudFormation deployment resource. The CloudFormation template defines this using the AWS::IAM::Role resource.
  • GitHub – A managed Git repository for storing source code and related files for this solution.
  • AWS CodeBuild – A managed service for running builds and tests or anything else you might run from a command line. In this solution, I am using CodeBuild to build two Lambda functions. One for getting data from a DynamoDB table and the other for posting data to this table. The CloudFormation template defines this using the AWS::CodeBuild::Project resource.
  • AWS CodePipeline – A managed service for orchestrating a software delivery workflow. In this solution, I am using CodePipeline to get source files from GitHub, build Lambda functions, and deploy them via CloudFormation. The CloudFormation template defines CodePipeline using the AWS::CodePipeline::Pipeline resource.

CodePipeline Deployment Pipeline

Here are the key components of the deployment pipeline defined in AWS CodePipeline. This pipeline is provisioned by the CloudFormation Stack that you launched. CodePipeline manages the continuous deployment capability for this serverless application. A snippet of the CloudFormation template that defines the deployment pipeline is shown in Listing 2.

  • GitHub – This is the Git repository that contains the Lambda functions, SAM template, and other files that are deployed on AWS.
  • AWS CodeBuild – Builds the Lambda functions by using the aws cloudformation command to generate the CloudFormation template based on the SAM template. Another CodeBuild project runs the AWS CLI command to insert data into the DynamoDB table as the last action in the pipeline.
  • AWS CloudFormation – The CloudFormation deploy provider generates a Change Set and then deploys the Lambda functions onto the AWS Lambda service.

Listing 2 – CodePipeline definition in CloudFormation template

Serverless Application Model Stack

The AWS SAM stack is launched by the CloudFormation deploy provider in CodePipeline. SAM provides serverless-specific capabilities and is an extension to CloudFormation that can also include CloudFormation resources. The CloudFormation deploy provider in CodePipeline is defined to call sam-http.yml. The template creates a CloudFormation stack that provisions a DynamoDB table, two Lambda functions, IAM permissions, and an API Gateway. A snippet of this SAM template is shown in Listing 2.

  • Amazon DynamoDB – Defines a DynamoDB table for storing information about cloud providers. Uses the AWS::DynamoDB::Table CloudFormation resource.
  • AWS Lambda – Two Lambda functions are defined in Node.js. One of these functions gets data from the DynamoDB table and the other posts data to this table. Uses the AWS::Serverless::Function SAM resource.
  • Amazon API Gateway – Create HTTP APIs and connect to different services. I am using the HTTP API feature to provide the “front door” to the data stored in the DynamoDB table. Uses the AWS::Serverless::HttpApi SAM resource.
  • AWS IAM – In the context of SAM, you often only need to define managed policies rather than defining detailed IAM permissions as you might do in CloudFormation. For example, I am using the SAM DynamoDBReadPolicy and
    DynamoDBWritePolicy managed policies to define permissions for the DynamoDB table.

Listing 2 – SAM Template that defines API Gateway, Lambda, and DynamoDB resources

Conclusion

In this post, you were able to launch a fully automated stack that deployed all of the resources to generate software architecture diagrams as code using AWS CloudFormation, AWS CodePipeline, and AWS CodeBuild.

All of the source code for this solution is located at https://github.com/PaulDuvall/cloudproviders/tree/master/webapp.

Photo by Marc Sendra Martorell on Unsplash

Stelligent Amazon Pollycast
Voiced by Amazon Polly