Monitor, alert, and discover sensitive data in your code for each commit. Get instant feedback on any commit you push to your repository. Spectral Bot can be installed directly on organizations and user accounts and grant access to specific repositories.

Integration Environment Variables

Setup Github App

First you will need to create a new GitHub app.

• If you are installing on an organization account use this link.
• If you are installing on your personal GitHub account use this link.

You then have to choose between installing on

• All repositories
  This applies to all current and future repositories owned by the resource owner.

• Only select repositories

Confirm your choice and move on to deploy the bot. After the bot is deployed you will need to return to the GitHub app settings and complete the setup. See the "Complete the GitHub App Setup" section below for details.

Deploy the bot

Deploy using one of the following methods:

Cloud Formation

Next we'll deploy the lambda to your AWS account. These permissions are required:

lambda:GetAccountSettings
cloudformation:DescribeStacks
iam:CreateRole
iam:DeleteRole
apigateway:POST
logs:CreateLogGroup
iam:PutRolePolicy

Go ahead and launch the stack

• To fill in the GITHUB_PRIVATE_KEY field, you'll need to generate a private key in your new GitHub app's settings and encode it with base64 without line-breaks:openssl base64 -A -in your-private-key.pem

• The options for the "Check Policy" field are

  • "Fail on any issue"
  • "Fail on critical only"
  • "Fail on high and above"
  • "Fail on medium and above"
  • "Fail on low and above"
  • "Always pass".

  This is for reference if you want to change it after deployment.

• If STRICT_MODE is set to true the status check considers on all issues found in the modified files, even if the issues are old. Otherwise, only new issues will be counted for the status.

Terraform

Deploy AWS resources using our Terraform module (set the integration_type parameter value to github).
NOTE: Terraform deployment supports deployment of GitHub bot 2.x versions and above.

Docker

Follow the deployment instructions on our DockerHub repo for the GitHub scanner.

Configuring Multiple GitHub Apps with a Single Instance of Spectral Bot

To enable multiple GitHub Apps with a single instance of the Spectral bot, you need to configure the multi_app.json file located in the root directory.

Steps for AWS Lambda Deployment

1. Download and Unzip Lambda Code:

   • Download the Lambda code as a .zip file.
   • Unzip the downloaded file.

2. Add multi_app.json:

   • Place the multi_app.json file into the unzipped directory.

3. Re-zip and Upload:

   • Select all the files within the unzipped directory.
   • Zip the files again.
   • Upload the new .zip file to AWS Lambda, ensuring the multi_app.json file is included.

Important Note

For AWS Lambda, ensure that both the frontend and backend Lambda functions contain the multi_app.json file within their respective deployments.

{
  "github": {
    "<app_id>": {
      "private_key": ""
    }
  }
}

Configuring the github Object in multi_app.json

The github object is a dictionary containing all the GitHub Apps. For each app, you must set the private_key, which is a base64-encoded string of the GitHub App's private key.

The <app_id> should contain the GitHub App ID, providing configuration for a specific application.

Optional Parameters

In addition to the private_key, you can configure the following optional parameters at the app level:

• check_failure_policy: Determines the failure policy for scans from this specific GitHub App. If not set, it defaults to CHECK_POLICY. Refer to the table above for possible values.
• spectral_dsn: Allows splitting scans using a different DSN for this GitHub App. It defaults to SPECTRAL_DSN.
• webhook_token: Allows using a webhook secret for this GitHub App. It defaults to GITHUB_WEBHOOK_SECRET.
• secret_vault: Enables storing secrets (webhook secret, app private key, spectral DSN) in a vault at the app level. When using a vault, you must specify all the keys for the webhook secret, GitHub App private key, and optional spectral DSN.

Example multi_app.json Configuration

Below is an example of a complete multi_app.json configuration:

{
  "github": {
    "<app_id>": {
      "private_key": "",
      "spectral_dsn": "",
      "webhook_token": "",
      "secret_vault": {
        "name": "aws_secrets_manager",
        "key_webhook_secret": "",
        "key_private_key": "",
        "key_spectral_dsn": ""
      }
    }
  }
}

Using vault

Instead of storing your secrets directly on the configuration of your Lambda, it is recommended for you to use a vault instead.

In order to use a vault, it is needed to set another environment variable called SECRETS_VAULT which contains the type of the vault you are using.

Available values:

• AWS secrets manager - aws_secrets_manager

Currently, we are supporting AWS secret manager, but in the future we are planning to support more vaults.

In your vault, make sure you set the following 3 secrets in your vault:

• Spectral_GithubBot_PrivateKey (for GITHUB_PRIVATE_KEY)
• Spectral_GithubBot_WebhookSecret (for GITHUB_WEBHOOK_SECRET)
• Spectral_Dsn (for SPECTRAL_DSN)

Overriding vault keys

To set up custom secret vault keys, use the following environment variables:

• VAULT_KEY_SPECTRAL_DSN - Spectral DSN key name, should be in the format of Spectral_Dsn-*
• VAULT_KEY_GITHUB_WEBHOOK_SECRET - GitHub app webhook secret, should be in the format of Spectral_GithubBot_WebhookSecret-*
• VAULT_KEY_GITHUB_PRIVATE_KEY - Private key, should be in the format of Spectral_GithubBot_PrivateKey-*

AWS Secrets Manager

If you are deploying using CloudFormation or Terraform, notice that the role created for the lambdas would give permission to perform secretsmanager:GetSecretValue action only for those 3 secrets.

Exclude repositories

To prevent the bot from scanning specific repositories you can supply the bot with the list of these "blacklisted" repo URLs.
To supply such list, you'll need to define two new environment variables:

1. S3_BLACK_LIST_BUCKET_NAME - the name of the bucket containing the blacklist file
2. S3_BLACK_LIST_OBJECT_KEY - the object key of the blacklist file

The blacklist file should be a text file containing a list of full URLs of repositories that has to be excluded (such as https://github.com/expressjs/express), with line breaks separating between them (each URL in a new line).

When the bot notices that the repo needed to be scanned is contained in the blacklist it would stop the execution and a check wouldn't be created.

If a problem occurs while fetching the blacklist, a failed check is created with an appropriate log in order to be able to know that a given push has not been scanned.

NOTE: Only single instance of the bot can be deployed to a region, if you wish to deploy several bot instances to the same region please contact customer support.

Complete the GitHub App Setup

After deployment is completed, go back to the GitHub app settings page.
In the Webhook section configure the Webhook URL, Webhook secret, and set the Webhook to active.

AWS Lambda

To build the webhook URL append /api/github/event to the ServiceEndpoint output from the stack:

https://<id>.execute-api.<region>.amazonaws.com/prod/api/github/event

Docker

To build the webhook URL append /events/github to the URL for your container:

https://<container-domain>/events/github

Save your changes to complete the process.

Monitoring

It is highly recommended monitoring the bot errors. It can be done easily if the bot is hosted using AWS lambda through CloudWatch alarms.

To create an alarm do the following:

1. In Cloudwatch, click on Create Alarm, and then Select Metric.
2. Select the Errors metric for the lambda and click on Select Metric.
3. Set the statistic to be Sum and select the required period for the threshold measuring.
4. Under conditions, choose static threshold, choose Greater, and set the threshold value to at least 1. Click Next.
5. Insert to Emails be to be alarmed. Click Next, name the alarm and click Next again.
6. Go over the summary, and if everything makes sense to you - click OK.

That's it 💪

Now all commits for the selected repositories are protected!

Upgrading the bot

For major updates please follow the breaking changes instructions.

With lambda

To upgrade the bot, you need to upload a new zip version to lambda under "Code" tab, make sure to upload the new versions to both frontend and backend lambdas.

With Docker

To upgrade the bot, you need to deploy a new Spectral GitHub bot image from docker hub.

Breaking changes

Learn about breaking changes that were introduced in the Spectral GitHub bot.

Upgrade from 1.x to 2.x

The Spectral GitHub bot is versioned.
We recommend upgrading from v1.x.x to v2.x.x due to major performance and resource utilization improvements.
Note: v1.4.2 is the latest version on the old generation of spectral bots.

v2.x.x changes:

• Improved performance.
• Improved logging.
• Fix bug - Decrease the number of lambda invocations.
• Fix bug - Decrease lambda invocations payload size.
• Fix bug - Issues with scanning first commit on a repository.
• Fix bug - Issues when running "re-check" scan on main branch.

Before upgrading to a new Spectral GitHub bot version, you should follow the required change instructions below:

Lambda and Docker:

Remove the empty environment variable if it exists. The new bot does not treat empty environment variables as non-existent.
Go to lambda backend/frontend -> Configuration tab -> environment variables -> edit and remove all the variables with empty values.

Environment Variables name changes:

• APP_ID -> GITHUB_APP_ID
• WEBHOOK_SECRET -> GITHUB_WEBHOOK_SECRET
• PRIVATE_KEY -> GITHUB_PRIVATE_KEY
• SHOULD_POST_REVIEW_COMMENTS -> GITHUB_SHOULD_POST_REVIEW_COMMENTS
• SHOULD_SKIP_GITHUB_CHECK -> GITHUB_SHOULD_SKIP_CHECK
• SPECTRAL_DSN_VAULT_KEY -> VAULT_KEY_SPECTRAL_DSN
• PRIVATE_KEY_VAULT_KEY -> VAULT_KEY_GITHUB_PRIVATE_KEY
• WEBHOOK_SECRET_VAULT_KEY -> VAULT_KEY_GITHUB_WEBHOOK_SECRET
• REPOS_BLACKLIST_BUCKET_NAME -> S3_BLACK_LIST_BUCKET_NAME
• REPOS_BLACKLIST_OBJECT_KEY -> S3_BLACK_LIST_OBJECT_KEY

Lambda only:

• Upload the new zip files frontend and backend
• Update runtime settings to work with Node.js 20.x on Lambda's frontend and backend.
• Under "runtime settings" change the handlers
  • On frontend from "src/frontend.app" to "index.handler"
  • On backend from "src/backend.app" to "index.handler"