DEV Community

Cover image for Build Your Own Lint Rules for Terraform with Shisho
Takashi Yoneuchi
Takashi Yoneuchi

Posted on

Build Your Own Lint Rules for Terraform with Shisho

tl;dr: Shisho is an open-source static code analyzer that lets you build your own lint rules for Terraform codes. You can find and refactor specific code patterns easily with a handy configuration language.

Building Linter / Static Analyzer is Too Hard

Every developer wants to avoid embedding issues in their software, while finding issues tends to be boring. So here's where a linter / a static analyzer come: they will be a great supporter of you and your team by detecting common bugs with pre-defined rules before the bugs are shipped to the world.

Sometimes, you will want to enforce custom rules for your code to standardize best practices specific to your team. When you want to prevent your team members from using uniform_bucket_level_access = true in google_storage_bucket resources like the following snippet, for example, you hope there's a flexible linter that lets you add custom rules quickly:

resource "google_storage_bucket" "test" {
  project  = var.project
  name     = "${var.project}-test"
  location = var.location
  uniform_bucket_level_access = true
  force_destroy               = true
}
Enter fullscreen mode Exit fullscreen mode

However, adding and maintaining custom rules is quite hard! You need to learn how to write custom rules for each programming language your team use, although different programming languages have different linters or analyzers, with different DSLs and APIs. This difficulty is one of the severe problems of standard linters / static analyzers.

Shisho: A Customizable Static Code Analyzer

Shisho, a lightweight static code analyzer, will help you build custom lint rules for your codebase. I'll explain what and how it is.

GitHub logo flatt-security / shisho

Lightweight static analyzer for several programming languages

shisho

shisho

GitHub Release GitHub Marketplace License Documentation Test Playground

Shisho is a lightweight static analyzer for developers.

Please see the usage documentation for further information.

demo

Try at Playground

You can try Shisho at our playground.

Try with Docker

You can try shisho in your machine as follows:

echo "func test(v []string) int { return len(v) + 1; }" | docker run -i ghcr.io/flatt-security/shisho-cli:latest find "len(:[...])" --lang=go
Enter fullscreen mode Exit fullscreen mode
echo "func test(v []string) int { return len(v) + 1; }" > file.go
docker run -i -v $(PWD):/workspace ghcr.io/flatt-security/shisho-cli:latest find "len(:[...])" --lang=go /workspace/file.go
Enter fullscreen mode Exit fullscreen mode

Install with pre-built binaries

When you'd like to run shisho outside docker containers, please follow the instructions below:

Linux / macOS

Run the following command(s):

# Linux
wget https://github.com/flatt-security/shisho/releases/latest/download/build-x86_64-unknown-linux-gnu.zip -O shisho.zip
unzip shisho.zip
chmod +x ./shisho
mv ./shisho /usr/local/bin/shisho
# macOS
wget https://github.com/flatt-security/shisho/releases/latest/download/build-x86_64-apple-darwin.zip -O shisho.zip
unzip shisho.zip
chmod +x ./shisho
mv ./shisho /usr/local/bin/shisho
Enter fullscreen mode Exit fullscreen mode

Then you'll see a…

Find Codes

First of all, Shisho enables us to run AST-aware code search over your code. Here's an example command which finds the occurence of uniform_bucket_level_access = true inside google_storage_bucket resource:

docker run -i -v $(pwd):/workspace ghcr.io/flatt-security/shisho-cli:latest find "
resource \"google_storage_bucket\" :[_] {
  :[...]
  uniform_bucket_level_access = true
  :[...]
}
" --lang hcl ./code.tf
Enter fullscreen mode Exit fullscreen mode

The command will make the following outputs in your console:

nYvlJB6

Here :[_] is an anonymous metavariable, which matches an arbitrary single node in AST (like a function call, identifier, and so on). Similarly, :[...] is an anonymous ellipsis metavariable, which matches zero or more nodes in AST. These operators are something like capture groups in regular expressions. They let you search over your code in a structured but flexible manner.

You can also define a rule, which includes a pattern and the explaination for it. The following YAML snippet is an example of rules describing the use of uniform_bucket_level_access is prohibited:

version: "1"
rules:
  - id: sample-policy
    language: hcl
    pattern: |
      resource "google_storage_bucket" :[_] {
        :[...X]
        uniform_bucket_level_access = true
        :[...Y]
      }
    message: |
      Our team policy prohibits the use of uniform bucket-level access.
Enter fullscreen mode Exit fullscreen mode

You can find patterns by executing shisho find path/to/rule.yaml path/to/search command, resulting in the following outputs:

Screenshot from 2021-08-23 19-08-44

This is how Shisho makes it possible to build your own lint rules for Terraform codes. You can use Shisho in the CI pipeline with your own rules, let alone your local machine. Please see Learn Shisho for further details.

Refactor Codes

Additionally, Shisho rules can include how detected code patterns should be fixed. The following YAML snippet describes a custom lint rule that suggests the use of uniform_bucket_level_access = true should be deleted:

version: "1"
rules:
  - id: sample-policy
    language: hcl
    pattern: |
      resource "google_storage_bucket" :[NAME] {
        :[...X]
        uniform_bucket_level_access = true
        :[...Y]
      }
    message: |
      Our team policy prohibits use of uniform bucket-level access.
    rewrite: |
      resource "google_storage_bucket" :[NAME] {
        :[X]
        :[Y]
      }
Enter fullscreen mode Exit fullscreen mode

Once this rule is run over your codes and the use of uniform_bucket_level_access = true is detected, Shisho suggests changes following the rule's rewrite section like:

Screenshot from 2021-08-23 19-07-30

Usecases

You can use Shisho for standardizing your codebase. In addition, it could be a means of conducting "security-as-code" or "policy-as-code"!

For instance, when you want to keep your team's EBS volumes encrypted, you can define a rule as follows:

version: '1'
rules:
  - id: 'unencrypted-ebs-volume'
    language: hcl
    message: |
      There was unencrypted EBS module.
    pattern: |
      resource  "aws_ebs_volume" :[NAME] {
        :[...X]
      }
    constraints:
      - target: X
        should: not-match
        pattern: |
          encrypted = true
    rewrite: |
      resource "aws_ebs_volume" :[NAME] {
        :[X]
        encrypted = true
      }
Enter fullscreen mode Exit fullscreen mode

When you want your colleagues to follow the naming convention for resources, the following rule will work well:

version: "1"
rules:
  - id: "invalid-resource-name"
    language: hcl
    message: |
      A resource was named badly.
    pattern: |
      resource  :[_] :[NAME] {
        :[...]
      }
    constraints:
      - target: NAME
        should: not-match-regex
        pattern: '"team1-.*"'

Enter fullscreen mode Exit fullscreen mode

The rule will report the result like:

Screenshot from 2021-08-24 00-34-46

Why Shisho?

"Modern Static Analysis: how the best tools empower creativity" explains that good code analyzers or linters are often interoperable, moldable, efficient, and community-driven and that Semgrep works well from these viewpoints. Semgrep is also grep-like (or sed-like) software that lets us find bugs with useful DSLs.

As for Shisho, it is at least interoperable (since it's open-sourced), moldable (though some efforts are needed; see issue #7). Moreover, Shisho is surprisingly efficient! Here's the result of a micro-benchmark of Semgrep, Comby (a similar tool), and Shisho1:

Tool Time Command
Comby (1.7.0) 263.1 ms time comby 'len(...)' '' parser.go -match-only &> /dev/null
Semgrep (0.62.0) 530.0ms time semgrep -e 'len(...)' --lang=go parser.go &> /dev/null
Shisho (0.1.2-alpha.2) 22.8ms time shisho find 'len(:[...])' --lang=go parser.go &> /dev/null

In fact, Shisho aims to refine the existing tools and to make it more feasible to run for large projects. You can use Shisho for your monorepo without hesitation. For your information, this speed is supported by Rust 2.

On the other hand, it's true that Shisho lacks some features of Semgrep and Comby. For instance, Semgrep has a feature to match patterns with type information while Shisho doesn't. Semgrep also has Semgrep Registry, in which you can share your own lint rules for the worldwide community. Now I'm making efforts to design and implement these features. Stay tuned!

Now What?

This article explained the usage of Shisho for Terraform codes, but Shisho is extending other language supports! Especially Dockerfile support will be shipped soon. You can follow @y0n3uchy to see the news on Shisho and star our GitHub project to encourage us :-)

Additionally, I'll release a SaaS which supports your Terraform development workflows with this engine. See https://shisho.dev/ for further details.


  1. Time is the average of 20 consecutive command executions. The measurement was run on Ubuntu 20.04.2 LTS with AMD Ryzen 5 3600 / 64GB RAM. The scan target was parser.go

  2. Both Semgrep and Comby is written in OCaml. 

Discussion (0)