03.03 Configure networking and install/configure a CI server

Resources

Here is a list of some of the scripts that are used in this sublesson.

Overview

  • Run a Ruby script that calls a CloudFormation script to configure deployment production line
  • CloudFormation configures a Configure Virtual Private Cloud (VPC) – including public and private subnets, a bastion host, Network Address Translation (NAT) nodes, IAM users, security groups, Route 53 Domain Name Service (DNS) and so on
  • Automatically launching and configuring Jenkins in a VPC – This included Network Address Translation (NAT) nodes, IAM users, security groups, Route 53 Domain Name Service (DNS) and so on. Run Jenkins jobs. Configure CI notifications. Navigate provisioning scripts and Jenkins. Maintain CI server data across instances. Log deployment production line activity

Prerequisites

Configure Route 53 Hosted Zone

  • From the Management Console, click on Route 53
  • Click the Create Hosted Zone button
  • Name the domain in the Domain Name field. Give it the name of a domain that you own and that you can use later
  • After entering the domain name, click the Create Hosted Zone button.
  • Select the checkbox next to the domain. You’ll see what’s called the Delegation Set in the Hosted Zone Details panel. Each of these four entries is a domain server that you will copy and paste in entering the domain servers with your domain registrar
  • Go into your domain registrar (Register.com, MediaTemple, GoDaddy, etc.) and for the domain name entered in the previous step, update your nameservers for your domain name to those listed in Delegation Set under Hosted Zone Details in the Route53 Console
  • Create CNAME records for the following and set the values to 1.1.1.1 (they’ll get updated through the automation later)
    • appdemo
    • honolulu
    • samplepipeline

Implement deployment production line steps

A list of the types of activities that might be performed as part of a deployment pipeline are described in this checklist.

[frontend-checklist name=”Deployment Production Line Steps”]

The jobs for the Honolulu Answers reference implementation are stored in jobdsl.groovy (or your corresponding Github fork). You’ll see more on jobdsl.groovy in the next sections.

Launch VPC and Jenkins

From the AWS CLI you created in in the Set up local development environment sublesson, type this command:

[sourcecode]cd ~/honolulu_jenkins_cookbooks
ruby create_vpc_and_jenkins.rb -k keyname -d domain_name[/sourcecode]

Replace keyname with your EC2 key pair name and domain_name with a domain you’ve configured in Route 53. An example is shown below.

[sourcecode]sudo gem install aws-sdk-core –pre[/sourcecode]

[sourcecode]ruby create_vpc_and_jenkins.rb -k stelligent07282014 -d devopsaws.org[/sourcecode]

It will take over 30 minutes to provision a new VPC and launch Jenkins inside this newly-generated VPC. As it’s provisioning, you’ll see messages similar to this:

[sourcecode]You’re creating a Honolulu Answers VPC and Jenkins instance in the us-west-2 region.
creating VPC………………
Giving the VPC a chance to generate security groups…
creating Jenkins server…………………………………….[/sourcecode]

You can find the Ruby script that runs these steps here: https://github.com/stelligent/honolulu_jenkins_cookbooks/blob/master/create_vpc_and_jenkins.rb

Configure CI notifications

Setup AWS SES

  • Setup AWS SES
  • Click Email Addresses and click Verify a New Email Address
  • Enter in an email address that you own and click Verify this Email Address
  • You’ll then get a verification email with a verification link that you need to click to confirm

Configure Jenkins Email Notifications to use SES

  • Edit the pipeline/jobs/jobdsl.groovy script using your desired text editor
  • Navigate to this snippet of code and change tech@stelligent.com to the email you setup using SES

[sourcecode]
publishers {
extendedEmail(‘tech@stelligent.com’, ‘$PROJECT_NAME – Build # $BUILD_NUMBER – $BUILD_STATUS!’, ”’$PROJECT_NAME – Build # $BUILD_NUMBER – $BUILD_STATUS:

Check console output at $BUILD_URL to view the results.”’) {
trigger(‘Failure’)
trigger(‘Fixed’)
}
}
[/sourcecode]

  • Save the .pipeline/jobs/jobdsl.groovy file changes and commit them to Github

Run Jenkins jobs

  • Go to your Jenkins server (Go to OpsWorks, find the same, click on the instance and select the ELB link)
  • Login to Jenkins with the default username and password (admin/admin)
  • Click the jenkins-test job
  • Click Build now link and wait about five minutes
  • Click the job-seed job
  • Click Build now link
  • Click the become-production-jenkins job
  • Click Build now link and wait a few seconds
  • Go to your browser and enter samplepipeline.yourdomain.com. For example, mine is samplepipeline.devopsaws.org

You’ll now get emails from Jenkins whenever a build fails or gets fixed

 

Navigate provisioning scripts and Jenkins

Maintain CI server data across instances

Store the Jenkins data in EBS and perform regular snapshots. When a Jenkins server goes down or is switched, attach an EBS volume from a snapshot.

Log deployment production line activity

One of the key benefits you can achieve when implementing Continuous Delivery is auditing all changes that occur as part of the deployment production line. When your entire software system is described in code and all canonical system changes get committed to a version-control repository, you can log and audit all changes that occur to the software system.

We do this in our cloud delivery systems using a Ruby API available on Github. Under the hood, this API accesses AWS’ NoSQL database called SimpleDB. Using this API with our Jenkins implementation, you can track the following activities:

  • Pipeline Instance ID – Unique id associated with a commit set
  • Commit SHAs – All commit SHAs associated to the pipeline instance
  • Stage – Deployment production line change of action
  • Action/Step – CI server job name on which the action occurs
  • Start time – of action
  • End time – of action
  • Success/Failure – of action

Here’s an example of using OpenDelivery to log pipeline activity (from trigger.sh)

[sourcecode]
# save instance id to SDB
ruby -v
gem install opendelivery –no-ri –no-rdoc
ruby -e ‘require "opendelivery"’ -e "OpenDelivery::Domain.new(‘$region’).set_property ‘$sdb_domain’,’$pipeline_instance_id’, ‘SHA’, ‘$GIT_SHA’"
ruby -e ‘require "opendelivery"’ -e "OpenDelivery::Domain.new(‘$region’).set_property ‘$sdb_domain’,’$pipeline_instance_id’, ‘started_at’, ‘$timestamp’"

# push instance id into file so we can load it into the environment
echo pipeline_instance_id=$pipeline_instance_id > environment.txt
[/sourcecode]

Leave a Reply