Provisioning Custom CodePipeline Actions in CloudFormation

A few weeks ago, I’d described the process I went through for provisioning CodePipeline using CloudFormation. As part of the provisioning of AWS CodePipeline, we were using the AWS CodePipeline CLI to provision a couple of custom actions. In my initial refactoring of the CodePipeline provisioning, I didn’t include the provisioning of the CodePipeline custom actions in CloudFormation. Since then, I’ve updated our Dromedary demo scripts to include this provisioning in CloudFormation so I’ll describe the process I went through in making these changes.

There are five types of action categories in CodePipeline: Source, Build, Deploy, Invoke and Test. Each action has four attributes: category, owner, provider and version. There are three types of action owners: AWS, ThirdParty and Custom. AWS refers to built-in actions provided by AWS. Currently, there are three built-in action providers from AWS: S3, CodeDeploy and ElasticBeanstalk. Examples of ThirdParty action providers include RunScope and Github. If none of the action providers suit your needs, you can define custom actions in CodePipeline. As mentioned before, you can do this using the CodePipeline CLI and – now – using CloudFormation.

The reason we needed to define custom actions in the first place was because we’re running custom behavior not provided “out of the box” by CodePipeline to launch environments, run static code analysis, automated tests and so on. For more information, see the StaticCodeAnalysis, CreateEnvironment, AcceptanceTest, InfrastructureTest and PromoteEnvironment action names in codepipeline-cfn.json.
The Resource Type we’re using in CloudFormation is the AWS::CodePipeline::CustomActionType. In the CloudFormation template, we set the Category, Provider, Version, ConfigurationProperties, InputArtifactDetails, OutputArtifactDetails, EntityUrlTemplate and ExecutionUrlTemplate for the custom action. This is explained in greater detail below.

• Category: For the first action, I’m using Build and for the second, I’m using Test.
• Provider: The custom-generated name for the Jenkins provider we created
• Version: For the Custom Action. I just used 1.
• ConfigurationProperties: Whether or not the custom action is required, queryable, etc.
• InputArtifactDetails: I’m providing the Maximum and Minimum counts for the input artifacts that get supplied to the custom action
• OutputArtifactDetails: I’m providing the Maximum and Minimum counts for the output artifacts that get supplied to the custom action
• EntityUrlTemplate: This is the Jenkins URL along with some Jenkins job-specific configuration
• ExecutionUrlTemplate: This is the Jenkins URL along with the last successfully executed Jenkins job-specific configuration.

I’ve defined two custom actions in this CloudFormation template. One uses the Build action category and the other, the Test category. Otherwise, they’re the same configuration. Ideally, and later, I’ll need to iterate on this using a CloudFormation DSL so that I’m not just copying/pasting the configuration.
You can find the custom actions CloudFormation template at codepipeline-custom-actions.json. This CloudFormation template is integrated with the rest of the CloudFormation templates that provision and configure this infrastructure so it’s not intended to be run on its own. For example, it needs for Jenkins to already be provisioned so that it can reference its unique provider configuration such as URL, name, etc. For instructions on running the Dromedary demo from CloudFormation, see CloudFormation Bootstrapping.

Troubleshooting Custom CodePipeline Actions for CloudFormation

It can be challenging to figure out what may have gone wrong when defining a custom action and using it in a pipeline. While the CodePipeline API will return errors, it’s not always easy to determine what you need to fix. Below, I’ve documented a checklist that you might use in troubleshooting any custom action errors.

• The Category property of AWS::CodePipeline::CustomActionType must be of one of the following values:
  • Source
  • Build
  • Test
  • Deploy
  • Invoke
• The Name property in ConfigurationProperties must be a valid configuration property for the Provider (e.g. Jenkins, etc.) you’re defining. For example, in codepipeline-custom-actions.json, ProjectName is a valid configuration property for the Jenkins CodePipeline job worker.
• When defining EntityUrlTemplate or ExecutionUrlTemplate in AWS::CodePipeline::CustomActionType, the ConfigNameHere (this is not the actual property name) property in {Config:ConfigNameHere} must match one of the property names in ConfigurationProperties of the same custom action
• Ensure the endpoints for EntityUrlTemplate and ExecutionUrlTemplate actually exist.
• With the exception of the Source stage, when using the custom action in a pipeline stage in CloudFormation (i.e the Actions property of the Stages property of a AWS::CodePipeline::Pipeline type), InputArtifacts must equal the OutputArtifacts value of a previously executed action. See CodePipeline Concepts for a visualization of this requirement.
• Run the following from your AWS CLI to see how your custom actions have been defined.
  aws codepipeline list-action-types --query actionTypes[*].id --output table
• Ensure you’re using valid Provider, Category, Owner and Provider property values. The same properties should be defined in your AWS::CodePipeline::CustomActionType that you’re using in the Actions property of the Stages property of your AWS::CodePipeline::Pipeline type
• Ensure the job worker for your provider has been configured correctly. See Create a Job Worker for Your Custom Action for more information.