Logic Rules (OPA)

Suggest Edits

Spectral has first-class integration with the OPA project (open policy agent), in the sense that it can use its logic facilities (doing away with the cruft of the "agent").

📘

Spectral's unique automatic policy routing

We reuse Spectral's own matches and parsers infrastructure to add-on Rego based logic in a seamless, performant way creating an intelligent scanning solution that doesn't require you to specify what or how to activate policies

IAC Spectral detector structure

IAC Detectors are composed of policies and queries run with the Spectral IAC engine against IAC files.

Each rule is a group of patterns, called a pattern_group, and is hierarchical (a pattern group can contain more patterns, but this is irrelevant while using OPA as you can create any logic you may want in one pattern using one or multiple policies.

The Spectral IAC detector structure YAML is similar to secrets detectors but using pattern_type: iac, for example:

- id: SCR_AWS_002
  name: Ensure all aws resource defined required tags keys are exists
  tags:
    - iac
    - other_tag 
  applies_to:
    - .*\.tf$
  severity: high
  pattern_group:
    patterns:
      - pattern: |
          package policies
          # opa policy 
          #     .
          #     .
          #     .
        pattern_type: iac

📘

Using your own tags names

For running IAC detectors with different tag than the iac tag (i.e 'spectral scan --include-tags other_tag'), add --engines iac to your Spectral scanning command. Otherwise, Spectral will not run the IAC engine.

Rego in Spectral

For those coming from a background in OPA and Rego, as a first impression, the general schema of a Spectral-compatible Rego policy looks like this:

package policies

Policy[result] {
  < LOGIC >
  result := {
    "id": "SCR_AWS_002",         # [!required] should must be same as Spectral rule ID.
    "keypath": "...",            # [!required] search expression to detect line numbers when needed
    "fix": "...",                # [?optional] instruction to how fix this finding
    "url": "https://example.com" # [?optional] a URL reference to documentation.
  }
}

Since Rego is non-opinionated, we've developed our own best practice for building detectors based on Rego.

Principles:

  • Every policy returns a result
  • Every result has a clear and strict schema, which Spectral needs in order to create a finding
  • Many policies can co-exist in the same pattern as long as each has the same id (in result).

Notes:

  • package polices is mandatory (this is how we know it's a rule related to Spectral). Originally OPA uses packages to optimize performance. However, we've opted to build our own more performant mechanism for namespacing, which is internal to Spectral. We do that by looking for the package polices package.
  • result := {...} and its assigned document are mandatory

OPA input objects

Policy is a piece of logic that returns results object if all conditions are satisfied for a given input object (in our case: IAC files content). So any policy should expect an input object.

The input object is a JSON list of objects. Each object represents the file content of a file with path that was included by apply_to. Each supported file format will be parsed to a JSON object.

Example for Rego input object

  1. Given a YAML file with this content:

    apiVersion: v1
kind: Secret
metadata:
  namespace: kube-system
  name: csi-s3-secret
stringData:
  accessKeyID: YOUR_ACCESS_KEY_ID
  secretAccessKey: YOUR_SECRET_ACCESS_KEY
  endpoint: https://www.double-you.com

    Will results input object as below:

    [
  {
    "apiVersion": "v1",
    "kind": "Secret",
    "metadata": { "namespace": "kube-system", "name": "csi-s3-secret" },
    "stringData": {
      "accessKeyID": "YOUR_ACCESS_KEY_ID",
      "secretAccessKey": "YOUR_SECRET_ACCESS_KEY",
      "endpoint": "www.double-you.com"
    }
  }
]

  2. Given Terraform file with this content:

    provider "aws" {
  region = "us-east-1"
}

resource "aws_s3_bucket" "my_bucket" {
 bucket = "my-bucket-name"
 acl    = "private"
 }

    Will results input object as below:

    [
  {
    "provider": { "aws": { "region": "us-east-1" } },
    "resource": {
      "aws_s3_bucket": {
        "my_bucket": { "acl": "private", "bucket": "my-bucket-name" }
      }
    }
  }
]

  3. Given JSON file with this content:

    {
  "name": "John Doe",
  "age": 30,
  "address": {
    "street": "123 Main St",
    "city": "Springfield",
    "state": "IL"
  }
  "hobbies": ["reading", "hiking", "cooking"]
}

    Will results input object as below:

    [
  {
    "name": "John Doe",
    "age": 30,
    "address": {
      "street": "123 Main St",
      "city": "Springfield",
      "state": "IL"
    },
    "hobbies": ["reading", "hiking", "cooking"]
  }
]

Building a Custom Spectral detector with OPA - Getting started

  1. Create an empty folder and cd into it.

  2. run spectral init to create a Spectral custom configuration environment.

  3. Create a new YAML file in .spectral/rules/my_rule.yaml (notice: my_rule can be any valid file name).

  4. Any Spectral rules YAML file starts with rules: object that contains a list of Spectral rules objects. i.e:

    rules:
- id: RULEID001
  name: ...
  tags:
  - ...
  severity: ...

  
- id: RULEID002
  name: ...
  tags:
  - ...
          .
          .
          .

    🦸‍♀️ Want to see a working Spectral file example? A complete example of a rules file will appear in .spectral/rules/simple.yaml after running spectral init.

  5. Add an file example with the finding you want to write a policy for in the root of the folder.

  6. For testing your rule: in the root folder, run spectral scan --engine iac --include <rule_id> or spectral scan --engine iac --include-tags <rule-tag>

Policy results

A results object will return from any policy that matched to finding, for example:
Given this simple policy that matches all AWS Terraform resources that have the tag production but have acl = public.

- id: ORG001
  name: Ensure all aws resources with tag production have ACL set to private
  tags:
    - iac
    - org
  applies_to:
    - .*\.tf$
  severity: high
  pattern_group:
    patterns:
      - pattern: |
          package policies
          Policy[result] {
            resource := input[i].resource[kind][name]
            regex.match("^aws_", kind) # any AWS resource
            resource.tags.Environment == "production"
            resource.acl != "private"
              
            result := {
              "id": "ORG001",
              "keypath": sprintf("%s[%s]", [kind, name]),
              "fix": sprintf("in `%s[%s]`, set 'ACL' to 'private' (instead of '%s')", [kind, name, resource.acl]),
            }
          }
        pattern_type: iac

Where run on the terraform file:

provider "aws" {
  region = "us-east-1"
}

resource "aws_s3_bucket" "my_bucket" {
  bucket = "my-bucket-name"
  acl    = "public"
  tags = {
    Name        = "my-instance"
    Environment = "production"
    Project     = "my-project"
  }
}

will return:

    "result": {
      "id": "ORG001",
      "keypath": "aws_s3_bucket[my_bucket]",
      "fix": "in `aws_s3_bucket[my_bucket]`, set 'ACL' to 'private' (instead of 'private')",
    }

Keypath

Use keypath to pointing a finding line number.
For example, the privies rule that matches all AWS Terraform resources with the tag production but has acl = public can have a few optional code lines you can point to. So choose what fits better in the detector context.

  1. Point to resource:

    "keypath": sprintf("%s[%s]", [kind, name]),

    will point to resource "aws_s3_bucket" "my_bucket" { line.

  2. Point to the ACL definition:

    "keypath": sprintf("%s[%s].acl", [kind, name]),

    will point to acl = "public"" { line.

  3. Point to the tag

    "keypath": sprintf("%s[%s].tags.Environment", [kind, name]),

    will point to Environment = "production" line.

Updated 10 months ago