tl;dr: Shisho is an open-source static code analyzer that lets you build your own lint rules for Terraform codes. You can find and refactor specific code patterns easily with a handy configuration language.
Building Linter / Static Analyzer is Too Hard
Every developer wants to avoid embedding issues in their software, while finding issues tends to be boring. So here's where a linter / a static analyzer come: they will be a great supporter of you and your team by detecting common bugs with pre-defined rules before the bugs are shipped to the world.
Sometimes, you will want to enforce custom rules for your code to standardize best practices specific to your team. When you want to prevent your team members from using uniform_bucket_level_access = true in google_storage_bucket resources like the following snippet, for example, you hope there's a flexible linter that lets you add custom rules quickly:
resource "google_storage_bucket" "test" {
project = var.project
name = "${var.project}-test"
location = var.location
uniform_bucket_level_access = true
force_destroy = true
}
However, adding and maintaining custom rules is quite hard! You need to learn how to write custom rules for each programming language your team use, although different programming languages have different linters or analyzers, with different DSLs and APIs. This difficulty is one of the severe problems of standard linters / static analyzers.
Shisho: A Customizable Static Code Analyzer
Shisho, a lightweight static code analyzer, will help you build custom lint rules for your codebase. I'll explain what and how it is.
flatt-security
/
shisho
Lightweight static analyzer for several programming languages
shisho
Shisho is a lightweight static analyzer for developers.
Please see the usage documentation for further information.
Try at Playground
You can try Shisho at our playground.
Try with Docker
You can try shisho in your machine as follows:
echo "func test(v []string) int { return len(v) + 1; }" | docker run -i ghcr.io/flatt-security/shisho-cli:latest find "len(:[...])" --lang=go
echo "func test(v []string) int { return len(v) + 1; }" > file.go
docker run -i -v $(PWD):/workspace ghcr.io/flatt-security/shisho-cli:latest find "len(:[...])" --lang=go /workspace/file.go
Install with pre-built binaries
When you'd like to run shisho outside docker containers, please follow the instructions below:
Linux / macOS
Run the following command(s):
# Linux
wget https://github.com/flatt-security/shisho/releases/latest/download/build-x86_64-unknown-linux-gnu.zip -O shisho.zip
unzip shisho.zip
chmod +x ./shisho
mv ./shisho /usr/local/bin/shisho
# macOS
wget https://github.com/flatt-security/shisho/releases/latest/download/build-x86_64-apple-darwin.zip -O shisho.zip
unzip shisho.zip
chmod +x ./shisho
mv ./shisho /usr/local/bin/shisho
Then you'll see a…
Find Codes
First of all, Shisho enables us to run AST-aware code search over your code. Here's an example command which finds the occurence of uniform_bucket_level_access = true inside google_storage_bucket resource:
docker run -i -v $(pwd):/workspace ghcr.io/flatt-security/shisho-cli:latest find "
resource \"google_storage_bucket\" :[_] {
:[...]
uniform_bucket_level_access = true
:[...]
}
" --lang hcl ./code.tf
The command will make the following outputs in your console:
Here :[_] is an anonymous metavariable, which matches an arbitrary single node in AST (like a function call, identifier, and so on). Similarly, :[...] is an anonymous ellipsis metavariable, which matches zero or more nodes in AST. These operators are something like capture groups in regular expressions. They let you search over your code in a structured but flexible manner.
You can also define a rule, which includes a pattern and the explaination for it. The following YAML snippet is an example of rules describing the use of uniform_bucket_level_access is prohibited:
version: "1"
rules:
- id: sample-policy
language: hcl
pattern: |
resource "google_storage_bucket" :[_] {
:[...X]
uniform_bucket_level_access = true
:[...Y]
}
message: |
Our team policy prohibits the use of uniform bucket-level access.
You can find patterns by executing shisho find path/to/rule.yaml path/to/search command, resulting in the following outputs:
This is how Shisho makes it possible to build your own lint rules for Terraform codes. You can use Shisho in the CI pipeline with your own rules, let alone your local machine. Please see Learn Shisho for further details.
Refactor Codes
Additionally, Shisho rules can include how detected code patterns should be fixed. The following YAML snippet describes a custom lint rule that suggests the use of uniform_bucket_level_access = true should be deleted:
version: "1"
rules:
- id: sample-policy
language: hcl
pattern: |
resource "google_storage_bucket" :[NAME] {
:[...X]
uniform_bucket_level_access = true
:[...Y]
}
message: |
Our team policy prohibits use of uniform bucket-level access.
rewrite: |
resource "google_storage_bucket" :[NAME] {
:[X]
:[Y]
}
Once this rule is run over your codes and the use of uniform_bucket_level_access = true is detected, Shisho suggests changes following the rule's rewrite section like:
Usecases
You can use Shisho for standardizing your codebase. In addition, it could be a means of conducting "security-as-code" or "policy-as-code"!
For instance, when you want to keep your team's EBS volumes encrypted, you can define a rule as follows:
version: '1'
rules:
- id: 'unencrypted-ebs-volume'
language: hcl
message: |
There was unencrypted EBS module.
pattern: |
resource "aws_ebs_volume" :[NAME] {
:[...X]
}
constraints:
- target: X
should: not-match
pattern: |
encrypted = true
rewrite: |
resource "aws_ebs_volume" :[NAME] {
:[X]
encrypted = true
}
When you want your colleagues to follow the naming convention for resources, the following rule will work well:
version: "1"
rules:
- id: "invalid-resource-name"
language: hcl
message: |
A resource was named badly.
pattern: |
resource :[_] :[NAME] {
:[...]
}
constraints:
- target: NAME
should: not-match-regex
pattern: '"team1-.*"'
The rule will report the result like:
Why Shisho?
"Modern Static Analysis: how the best tools empower creativity" explains that good code analyzers or linters are often interoperable, moldable, efficient, and community-driven and that Semgrep works well from these viewpoints. Semgrep is also grep-like (or sed-like) software that lets us find bugs with useful DSLs.
As for Shisho, it is at least interoperable (since it's open-sourced), moldable (though some efforts are needed; see issue #7). Moreover, Shisho is surprisingly efficient! Here's the result of a micro-benchmark of Semgrep, Comby (a similar tool), and Shisho1:
| Tool | Time | Command |
|---|---|---|
| Comby (1.7.0) | 263.1 ms | time comby 'len(...)' '' parser.go -match-only &> /dev/null |
| Semgrep (0.62.0) | 530.0ms | time semgrep -e 'len(...)' --lang=go parser.go &> /dev/null |
| Shisho (0.1.2-alpha.2) | 22.8ms | time shisho find 'len(:[...])' --lang=go parser.go &> /dev/null |
In fact, Shisho aims to refine the existing tools and to make it more feasible to run for large projects. You can use Shisho for your monorepo without hesitation. For your information, this speed is supported by Rust 2.
On the other hand, it's true that Shisho lacks some features of Semgrep and Comby. For instance, Semgrep has a feature to match patterns with type information while Shisho doesn't. Semgrep also has Semgrep Registry, in which you can share your own lint rules for the worldwide community. Now I'm making efforts to design and implement these features. Stay tuned!
Now What?
This article explained the usage of Shisho for Terraform codes, but Shisho is extending other language supports! Especially Dockerfile support will be shipped soon. You can follow @y0n3uchy to see the news on Shisho and star our GitHub project to encourage us :-)
Additionally, I'll release a SaaS which supports your Terraform development workflows with this engine. See https://shisho.dev/ for further details.
Top comments (0)