config-lint: Up and Running

How to leverage config-lint to validate JSON and YAML based configurations.

Configuration Validation with config-lint

Stelligent config-lint is an open source command line tool to lint configuration file in a variety of formats, including JSON, YAML, Terraform, and Kubernetes. Config-lint offers a lightweight method to provide custom validation for unique compliance guidelines, as well as adding an opinionated list of built-in rules to ensure configuration meets best practice in design and security. In addition, config-lint enables developers to validate configuration files more quickly, consistently, and effectively.

Running config-lint

There are several ways to install config-lint, but in this example we will use Docker. Other methods of installation are listed at the GitHub repository. We’ll set up a demo environment in Cloud9 to demonstrate installation, configuration, and execution from beginning to end. To learn more about AWS Cloud9, check out our blog post, Automating AWS Cloud9.

Set up Cloud9

  1. Go to the AWS Cloud9 Console
  2. Select Create environment
  3. Enter a Name and Description
  4. Leave all Environment settings as default.
    • Environment type: Create a new instance for environment (EC2)
    • Instance type: t2.micro
    • Platform: Amazon Linux
    • Cost-saving setting: After 30 minutes (default)
    • IAM role: AWSServiceRoleForAWSCloud9
  5. Select Next step
  6. Review best practices and select Create environment

Installing config-lint with Docker

Your Cloud9 environment’s Amazon Linux instance will already have Docker installed.

Pull the latest config-lint release with docker:

docker pull stelligent/config-lint:latest

Docker needs access to the configuration files, so we need to mount the directory to the running container. Other than that, config-lint is ready to run. Execute the version command to demonstrate the docker run command:

docker run -v $(pwd):/~ stelligent/config-lint --version

We can save this as an alias so that we don’t have to type out the whole command each time:

alias config-lint="docker run -v $(pwd):/~ stelligent/config-lint"

Again, we can validate the setup with the following:

config-lint --version

Out of the box, config-lint will run a builtin rule set against Terraform. Let’s clone an example repository to run against:

git clone https://github.com/stelligent/config-lint

Run the builtin Terraform rules against the cloned example files:

config-lint -terraform config-lint/example-files/demo-resources

The result should return the expected result, with a single failure.

[
  {
    "AssertionMessage": "should have property server_side_encryption_configuration",
    "Category": "resource",
    "CreatedAt": "2020-04-06T22:54:55Z",
    "Filename": "example-files/demo-resources/s3-bucket.tf",
    "LineNumber": 1,
    "ResourceID": "b1",
    "ResourceType": "aws_s3_bucket",
    "RuleID": "S3_BUCKET_ENCRYPTION",
    "RuleMessage": "S3 bucket should be encrypted",
    "Status": "FAILURE"
  }
]

Opening the named file, example-files/demo-resources/s3-bucket.tf, we can see the invalid resource. In that example file, the correct configuration is commented out. Uncomment the configuration for server_side_encryption_configuration and rerun config-lint. Now config-lint will return [], no warnings or failures found.

Developing Custom Rules

In addition to validating against our built-in ruleset, config-lint provides a simple and powerful way to add custom rules for any JSON, YAML, Terraform, or Kubernetes configuration. Custom rules allow for checks against the unique requirements of a use case.

Example Terraform Rule

For example, with a simple rule, we can check for a tag on a given Terraform resource.

version: 1
description: example
type: Terraform
files:
  - "*.tf"
rules:
  - id: BUCKET_TAG
    message: S3 buckets require example tag
    resource: aws_s3_bucket
    assertions:
    - key: "tags[].Department | [0]"
      op: in
      value: Operations,Sales,Marketing,Engineering
    severity: WARNING

Each rule requires a JMESPath key that it will use to search resources. The expressions can be tricky to get right, so config-lint provides a -search option which takes a JMESPath expression. The expression is evaluated against all the resources in the files provided on the command line. The results are written to stdout.

This example will scan the example terraform file and print the “ami” attribute for each resource:

./config-lint -rules example-files/rules/terraform.yml -search 'ami' example-files/config/terraform.tf

By specifying “-search”, the rules files are only used to determine the type of configuration files. The files will not be scanned for violations.

Creating a Custom Rule

Let’s demonstrate creating this custom rule with our Cloud9 Environment, given config-lint installed with Docker.

Open a new file under the home directory, named custom-rule.yml, and paste the above rule.

Execute this rule against an example resource, downloaded with our clone earlier:

config-lint -rules ~/custom-rule.yml config-lint/example-files/demo-resources

The command should return this expected result with a single warning:

[
  {
    "AssertionMessage": "tags[].Department | [0](invalid) should be in Operations,Sales,Marketing,Engineering",
    "Category": "resource",
    "CreatedAt": "2020-04-06T22:51:06Z",
    "Filename": "example-files/demo-resources/s3-bucket.tf",
    "LineNumber": 1,
    "ResourceID": "b1",
    "ResourceType": "aws_s3_bucket",
    "RuleID": "BUCKET_TAG",
    "RuleMessage": "S3 buckets require example tag",
    "Status": "WARNING"
  }
]

Opening the named file, example-files/demo-resources/s3-bucket.tf, we can correct the resource and rerun config-lint. Now config-lint will return “[]”, no warnings or failures found.

Example YAML Rule

Custom rules also give the ability to make generic checks against any JSON or YAML configuration file. For example, the following rule will check the configuration of two resources in an arbitrary YAML file. We’ll define the resources “widget” and “gadget” and set a rule for each.

version: 1
description: Rules for generic YAML file
type: YAML
files:
  - "*.config"
resources:
  - type: widget
    key: widgets[]
    id: id
  - type: gadget
    key: gadgets[]
    id: name
rules:
  - id: WIDGET_NAME
    message: Widget needs a name
    severity: FAILURE
    resource: widget
    assertions:
    - key: name
    op: present
  - id: GADGET_COLOR
    message: Gadget has missing or invalid color
    severity: FAILURE
    resource: gadget
    assertions:
    - key: color
      op: in
      value: red,blue,green

Once again, we can run this rule against our configuration with the following command format:

config-lint -rules example-yaml-rule.yml configuration-file.yml

These are just a few examples. Use config-lint custom rules to meet your own enterprise needs. Whether compliance requires specific tags, resources need special configuration, or you’ve created custom configuration files, config-lint provides a mechanism to define rules and validate the configurations unique to you.

config-lint Roadmap

While the beta versions have been publicly available for over a year, our new 1.0 release includes support for Terraform 0.12, expanded platform support including a Windows executable and ARM CPU support, and various bug fixes and quality improvements.

Going forward, we will continue to build upon and improve config-lint. Over the next 3 months, expect to see continuous automated releases, improvements to the documentation and wiki, increased AWS Terraform built-in rule coverage, and TF11 deprecated. Then over the next 6 months, expect to see additional support for builtin Terraform rules, as well as updated support for Kubernetes.

Contributions

config-lint is an open-source project; contributions and feedback are welcome! Feel free to read our code and contribute at our GitHub repository. Please submit Issues for debug fixes, features requests, or questions in GitHub.

Stelligent Amazon Pollycast
Voiced by Amazon Polly