re:Invent is just around the corner (next week!), which means a bunch of new services and features will be announced, and they probably won’t have CloudFormation support for a few weeks. As the biggest CloudFormation fanboy of all time, this is probably the most difficult time of the year for me, as I want to use all the new goodies but feel kinda icky not having everything codified in a CloudFormation template. 
This year I’ve decided to prepare for re:Invent by putting together some templates and stub code to set up Custom Resources using Lambda. While they don’t do anything interesting by themselves, what we’ll cover today is a great starting point for creating your own CloudFormation custom resources. 
CloudFormation Custom Resources are a neat feature that let you configure your own custom actions that can be triggered by CloudFormation when it is building your stacks. It can be easily configured to trigger a Lambda function. You can use custom resources to do much more complicated actions than CloudFormation normally allows, which is great if you have complex conditional logic you want to remove from your template, if you need to interact with non-AWS systems during CloudFormation stack launches, or most importantly for me, want to leverage newly announced AWS features before CloudFormation support exists, but still want all your AWS resources to be in code in your source control repo.  
There are two parts to using a custom resource: the lambda function that executes whatever you need, and the Custom Resource call in your CloudFormation template. Let’s look at each. 
Creating Your Custom Resource Lambda 
Before you can call your custom resource, you need to create it, and we can use CloudFormation to do that! When creating any Lambda function with CloudFormation, there are three things you need to do: 

  • Write your Lambda function code and get it into S3 
  • Create your Lambda Function CloudFormation definition 
  • Create your Lambda Function IAM Role CloudFormation definition 

The first thing you’ll need to do is define your function. There’s been a lot written about how to write Lambda functions, so I read a bunch of it and with some help from some other Stelligentsia put together starter functions in NodeJS, Java, and Python to make it easy to get going.   
Once you have your function code write, you’ll need to get it into Lambda. You can define your Lambda function code in line in your CloudFormation template, but much like dealing with EC2 user-data, this can get very messy very quickly. It’s much easier to just zip up your function code, store it in S3, and then pass the S3 location as a parameter. You’ll probably want to set up a pipeline to move the function code from your source code repo into zips in S3, but the actual code in that pipeline will be pretty simple:  

 Then when you define your template, include parameters for specifying that location   

 Then you can configure your Lambda resource to look in that location. 

Chances are you’re going to want to do something in your AWS account with your custom resource, so your Lambda function will need an IAM role for it to be able to do that. That’s easy enough to configure:   

These snippets highlight the important bits, but they are not complete. To view the full template, in both JSON or YML, check out the github repo. 
Calling your Custom Resource 
Once your custom resource is created, you’ll need to call it. The trickiest part of this is getting the ARN of your Lambda function into the template so that CloudFormation knows who to call. If you’re creating and consuming your resource in the same CloudFormation tempalte, this is easy: you just need to extract the ARN from the Lambda function resource with GetAtt: 

However, if you’re going to be calling this functionality from a bunch of different templates, you probably don’t want to create a separate lambda function for each one that does the same thing. In this case, you’ll want to define your Custom Resource in its own template, and have it spit out the ARN as an output: 

At this point, you could just copy and paste that ARN and hardcode it in the template that consumes it. Or you can then pull that ARN programmatically with the AWS CLI and pass it as a parameter: 

Again, these snippets just show off the important bits, for the full templates check out the github repo. 
Gotchas 
While custom resources allow you to execute whatever arbitrary code you need during your CloudFormation stack creation, update, or delete, they do have a couple of gotchas you should be aware of.  
One gotcha is going to be that CloudFormation will not execute a stack update unless it detects that the template or parameters have been changed. If your custom resource will do different things on each execution (for example, generating a random number) and you would like to issue updates to trigger that functionality without updating the stack, CloudFormation will decline to update the stack. To work around this, you can have a dummy parameter that you update that is not used by the resources, and can be changed as needed. A very common parameter to include is a timestamp, which will always be different, and has the added bonus of occasionally being useful. 
The other thing you should be aware of when starting with custom resources (or, more generally, with Lambda functions) is that everything printed to the console or logger will end up in the CloudWatch logs for that function. While most of the time this is super helpful, if you’re handling sensitive information in your custom resource code, you’ll want to make very sure that you’re not emitting that in places you don’t want to. 
Finally, while developing custom resources, it becomes very easy to get yourself into situations where things don’t delete cleanly. While coming up with these examples, on multiple times I deleted a custom resource function before I cleaned up all the stacks that used it – this caused stack deletions to fail. Also, I created several resources before implementing the proper delete logic to clean them up. While not the end of the world, they did clutter up the CloudFormation console and required that I wait for delete operations to time out and then retry them to get stuff cleared out. 
Conclusion 
Custom resources are an easy way to handle complicated logic inside your CloudFormation templates. Setting them up can be a bit tricky, but hopefully the sample code provided will help guide you in the right direction. Also, keep an eye out for more posts using custom resources next week during re:invent, and sample code for more languages in the future! 
Also, big thanks to fellow Stelligentsia Dan Ghadge and Larry Hitchon who helped out with developing the Java and Python functions!

Stelligent Amazon Pollycast
Voiced by Amazon Polly