AWS allows you to build enormous and complex cloud infrastructures in a matter of hours. With the ability to create resources so easily, sometimes it can be hard to manage all those resources. If only there were a simple but powerful tool that could manage it all. Cloud Custodian (a.k.a C7N) is a Python CLI tool that gives you powerful account management capabilities with a simple config file. Cloud Custodian can help you manage your AWS account using a simple policy config file and time-based or event-based Lambdas. The config files (YAML formatted) allow you to define policies for everything from tag compliance to backups. Define policies for a wide variety of management activities, including garbage collection to encryption.
Cloud computing has made creating and managing web resources insanely easy, quite possibly too easy. You can now spin up quite a few computing, database, and storage resources with the click of a button or the stroke of a return key. However, if you use a company account, you likely spin up those resources often for demonstration and testing purposes, without considering the cost or clutter you might be creating along with it. This was “the problem” at Capital One when they created this very powerful tool for managing the cleanup of your superfluous cloud resources. Capital One started developing Cloud Custodian in July 2015 and open-sourced the tool in April 2016.
Cloud Custodian’s feature-set has grown exponentially with it’s popularity because they’re very good about responding to feature requests. It’s now grown to the point where there’s not much in the AWS world you can’t do with it. Here’s a short list of some things you might be surprised it can do:

  • Encryption
  • Backups
  • Garbage Collection
  • Unused Resources
  • Off-hours
  • Tag Compliance
  • SG Compliance

Odds are though, you’re considering Cloud Custodian for it’s namesake: cleaning up your AWS account; resource/cost management during off-hours; and overall garbage collection. True to it’s name, this is where Cloud Custodian excels. With a relatively simple configuration file you can tidy and trim your AWS account and keep it that way as you grow your business.
Here’s a very basic example custodian.yml file that stops all EC2 instances tagged with Custodian:

policies:
  - name: stop-instances
    resource: ec2
    filters:
      - "tag:Custodian": present
    actions:
      - stop

Cloud Custodian is great for mid to large sized companies that give a large number of their employees full access to a company AWS account. Naturally, their account quickly becomes cluttered with dozens of CloudFormation stacks, VPCs, old test instances, and Lambda functions. Here at Stelligent we have an AWS Labs account for exploring and testing in AWS. We use Cloud Custodian to clean up old testing resources based on age and resource tags. 

Your Cloud Custodian Strategy

singlenodedeploy.pngSource

As of today, there’s not much Custodian can’t do in your AWS account so it’s good to explore what Cloud Custodian can do for you before deciding on your overall strategy. Here are four common use-cases:

  • Automatic clean-up: Using the mode property, you can run actions in response to a variety of CloudWatch Events. Read more
  • Monitoring your environment: This is one of my favorite features. Custodian generates CloudWatch metrics by default so it’s easy to throw together dashboards that give you full visibility into what is being managed by Custodian and what isn’t. It’s hard to get good visibility in a vast system like AWS. Read more
  • Stopping Resources during Off-hours: Custodian makes it very easy to set up Off-hours for your resources based on tags. Below is an example. Read more

    policies:
  - name: offhours-stop
    resource: ec2
    filters:
      - type: offhour
        tag: downtime
        onhour: 8
        offhour: 20
    actions:
      - stop
  • Tag-compliance: One of the most common use cases for Custodian is tag-compliance. You can manage tag-compliance policies for your entire account in a single config file. You can even check it into version control. And, if you’re really ambitious, you can create a pipeline to watch your version control system that runs custodian for you so you don’t have to activate virtualenv on your personal machine every time you want to make a change. Read more

Prereqs and Pro Tips

Cloud Custodian is very well documented, so if you’re excited to start taking out the digital trash in your AWS account there’s no better place to start than their website and documentation. There are a few things to keep in mind before diving head-first into the cloud equivalent of the custodial arts:

Prereqs

At Stelligent we’re a tiny bit obsessed with one-command solutions. I have to admit, I cringed a little when I saw that Cloud Custodian took 3 or more commands to install and run. In the Getting Started section of the docs, Custodian requires you to have python, pip and virtualenv installed before you can even install Cloud Custodian. Then, once you activate virtualenv to install and run it the first, you’ll need to re-activate the virtualenv every time you want to run it in the future. That’s why I recommend using a Pipeline or CloudFormation template which brings us directly to our pro tips:

Pro Tip #1 – Minimalist Custodian

The easiest way to get started cleaning up your AWS account with Custodian is to go through your account and tag everything you want to keep with something like “NoCustodian”. Then, set

policies:
  - name: stop-instances
    resource: ec2
    filters:
      - "tag:Custodian": present
    actions:
      - stop

Click the button below to launch an example CloudFormation Stack that boots an EC2 instance and then uses Custodian to stop the instance.

Pro Tip #2 – Don’t piss off your co-workers

The first thing you’ll be tempted to do when implementing Cloud Custodian is terminate all the old and un-used resources in your account. Just be sure all the relevant parties in your company know what you’ll be terminating and when.

Pro Tip #3 – Use a Pipeline

Setup a CodePipeline that allows you to keep your custodian.yml file in source control and re-run it with CodeBuild every time you commit a change.

In Summary

If you need better visibility and automated management of your AWS account, Cloud Custodian has lots of helpful features that are easy to manage in a single config file. If you aren’t already a python developer, I recommend setting up a CloudFormation template or Automated Pipeline to manage changes to your. You can use the launch button in Pro Tip #1 to see an example of Custodian in a CloudFormation template. Keep a look out for a future blog post with a detailed example of a fully-automated Custodian Pipeline.
Links
Custodian Website
Custodian Docs
Custodian GitHub
Videos
AWS This Is My Architecture: Cloud Custodian
Cloud Custodian @ AWS re:Invent
Cloud Custodian @ Serverlessconf

Stelligent is hiring! Do you enjoy working on complex problems like figuring out ways to automate all the things as part of a deployment pipeline? Do you believe in the “one-button everything” mantra? If your skills and interests lie at the intersection of DevOps automation and the AWS cloud, check out the careers page on our website.

Stelligent Amazon Pollycast
Voiced by Amazon Polly