Enterprise Scale CI/CD with Pipeline Factory on AWS
Find this blog interesting? Make sure you register and join us on May 12-13 at the AWS Summit Americas for a demo.
With the adoption of modern application architectures, many enterprises find they need the capability to deliver hundreds or even thousands of distinct applications and their CI/CD pipelines. Scaling the capability to deliver CI/CD pipelines brings a new set of questions for your DevOps teams:
- How do you develop and deploy CI/CD pipelines quickly and on-demand?
- How do you ensure consistency and quality in delivery workflow?
- How can you enforce regulatory compliance and security controls?
- How do you achieve governance while empowering developers?
By defining CI/CD pipelines as software products themselves, DevOps teams gain the capability to develop and publish validated pipelines. You can then define, build, and publish standardized pipelines that enforce quality, security, and governance throughout the software delivery lifecycle. Product and development teams are empowered to deploy validated pipelines on-demand via self-service catalogs without involving operations teams.
CI/CD Pipeline Products
A CI/CD pipeline consists of a variety of stages and actions that orchestrate software delivery from source to production. Utilizing AWS CodePipeline, a fully managed CI/CD service, you can automate your pipelines for fast and reliable application deployments. AWS CloudFormation, allows you to codify the workflow and stages of your CI/CD pipeline. With codified pipelines you can create a framework of standardized CI/CD pipelines. Templates can be built for each type of pipeline pattern needed. Templates can also be parameterized to allow for optional pipeline configurations. This template creates a deployable Pipeline Product for consumption by development or product teams.
DevOps teams can design CI/CD patterns that will be utilized for repeatable deployment of application products. Those patterns include standardized sets of stages needed to source, build, test, and deploy the application products and are parameterized to allow for optional configurations of the pipeline. The defined pipeline stack and its configuration are codified in the form of a CloudFormation template. That template can be stored in CodeCommit for version control allowing iteration of the pipeline pattern as needed.
Validating and Publishing Pipeline Products
Having codified the Pipeline Product, an enterprise should now treat this as any other software product and develop a pipeline to validate (test) and publish (deploy) for consumption. With each iteration or change to the Pipeline Product, the pipeline will automatically process the new template, automating updates to Pipeline Product, and making the latest versions available for consumption by product and development teams.
This pipeline should conduct static analysis of the template utilizing tools like Stelligent’s cfn_nag and AWS CloudFormation Guard to validate that pipelines are enforcing security and governance controls as well as meeting enterprise standards for infrastructure configuration. This stage of the pipeline should also validate that all required testing suites are included in the pipeline ensuring that enterprise quality standards are enforced.
Next, you can extend validation by executing an Operational Validation stage. In this stage, the Pipeline Product is deployed into an isolated test AWS environment. Utilizing AWS Config, you can ensure that the operation state of the Pipeline Product is compliant with the security, governance, and quality standards your enterprise has set.
Finally, with a validated template and operational environment, the Pipeline Factory will publish the CloudFormation template to AWS Service Catalog as a Pipeline Product for consumption by product and development teams.
By codifying Pipeline Patterns, DevOps teams can adopt a model where CI/CD pipelines are built as a product for consumption by internal development teams. Product and development teams can now deploy new CI/CD pipelines on-demand in minutes instead of spending hours configuring pipelines manually. Developers can focus on developing new features and applications instead of worrying if they have CI/CD built right. Configuration of new pipelines is no longer a bottleneck to software delivery velocity.
For DevOps teams, Pipeline Factory provides a framework that allows them to standardize on common CI/CD patterns. This allows DevOps teams to achieve a high level of quality for software delivery systems. Enterprises can also define and enforce security and governance controls in all CI/CD pipelines. The result is delivery of higher quality application code, secure and compliant infrastructure, and incredible agility to provide software delivery capabilities for your enterprise.
If you’re ready to see how Stelligent’s Pipeline Factory can help you scale your application delivery while achieving unprecedented levels of quality, reach out to one of our Solutions Architects today for an analysis of your current CI/CD architecture.
Stelligent Amazon Pollycast