DEV Community

Ilya Rubinchik
Ilya Rubinchik

Posted on

Your terragrunt (or terraform) plan is 4,000 lines. Only two of them matter.

#devops #opensource #aws #terraform

You know the ritual.

terragrunt run --all -- plan
Enter fullscreen mode Exit fullscreen mode

Then you scroll. Past forty units of Refreshing state…. Past the ninth
identical count instance. Past a tags_all.LastModified that changes on every
single run because your CI stamps a timestamp into it. Somewhere in there are
the two lines you actually needed to see — probably the # forces replacement
on a database.

You scroll back up. You lose it. You pipe it to a file and grep for must be
replaced. You approve anyway, because it's 6pm.

I got tired of that, so I wrote tgsieve.

What it does

It runs the plan for you, reads the structured output instead of the prose,
throws away the noise you declared as noise, collapses everything that repeats,
and prints what's left.

DESTROY / REPLACE (1)
  envs/prod/a
    ± aws_db_instance.main
        engine_version  "14.7" → "15.3"   forces replacement

UPDATE (5)
  5 units  envs/dev/a, envs/dev/b, envs/prod/a, +2 more
    ~ null_resource.pin
        triggers.region  "eu-central-1" → "us-west-2"

SUMMARY  ±1 replace  ~5 update
  severity: 1 high, 5 medium
  hid 214 attributes across 3 rules (--explain to see them)
Enter fullscreen mode Exit fullscreen mode

That's five units of a real terragrunt plan — the same run terraform prints as
several hundred lines.

The report nests three deep — where, then what, then which fields:

UPDATE (5)
  envs/prod/c                            ← the unit, said once
    ~ aws_s3_bucket.this                 ← the resource
        tags_all.entity  "tgb" → "tgc"   ← the attributes that changed
Enter fullscreen mode Exit fullscreen mode

A change that's identical across units replaces the directory with the set it
covers, so the first column always answers the same question: where.

It doesn't scrape text

This matters, because the obvious implementation is fragile garbage.

You might reach for terragrunt run --all -- plan -json. It doesn't work:
terragrunt forwards terraform's own NDJSON straight through, so lines from units
running in parallel interleave with no way to tell them apart.

So tgsieve asks terragrunt for machine-readable artifacts and reads those:

What Flag it passes What it gets
per-unit plans --json-out-dir one tfplan.json per unit — the full terraform show -json document
live progress --log-format json NDJSON events tagged with working-dir, so failures surface the moment they happen
run report --report-file/--report-format json per-unit result and duration, including units that failed before producing a plan

From each plan it computes a real attribute-level diff — flattening
before/after/after_unknown into dotted paths, honouring replace_paths,
sensitivity, and unknown subtrees. No regexes over ~ resource "aws_...".

OpenTofu works too — terragrunt defaults to tofu, and --tf-path /
TG_TF_PATH picks explicitly.

Nothing is hidden until you say so

The whole tool is one .tgsieve.yaml, looked up from the working directory
upwards and merged, nearer files winning. tgsieve init writes a starter one.

version: 1

extends: [builtin/aws-tags]   # curated rule sets, opt-in

hide:
  unchanged_units: true   # units with nothing left to say become a count
  reads: true             # data sources resolved during apply: they create nothing

ignore:
  - name: tag churn
    attrs: ["tags.LastModified", "tags.git_commit", "tags_all.*"]

  - name: waiting on the provider fix
    type: aws_ecs_service
    attrs: ["capacity_provider_strategy.*"]
    expires: 2026-12-01     # after this date the rule stops hiding, loudly

  - name: dev is not interesting
    unit: "envs/dev/**"
    attrs: ["*"]

never_hide:
  actions: [delete, replace]
Enter fullscreen mode Exit fullscreen mode

Three things I'd want to know before trusting a tool that hides my
infrastructure changes:

  • A resource only disappears when every one of its attributes was hidden. One survivor keeps the whole resource on screen.
  • An attribute that forces replacement is never hidden, whatever the rules say. Destroys and replacements can't be silenced by default either.
  • --explain shows every hidden attribute and the rule that hid it, and the footer always states how much was hidden. The counts count real resources, not rendered blocks.

Suppressions that expire

That expires: 2026-12-01 is my favourite line in the config. Past that date
the rule stops hiding anything and the report names it:

1 rule expired and no longer hides anything: waiting on the provider fix
Enter fullscreen mode Exit fullscreen mode

It fails open, on purpose. A suppression that quietly outlives its reason is
exactly the failure this tool exists to prevent. So the lapse restores the
changes rather than silently continuing to swallow them.

The small readability things

These are the ones that made me realise how much terraform's renderer was
costing me.

Sets are compared by members, not positions. Terraform renders sets as
arrays, so a set that comes back in a different order looks like every index
changed at once. tgsieve says what actually happened:

input.cidrs  reordered (4 items, same members)

input.cidrs  - "10.0.2.0/24"
input.cidrs  + "10.0.9.0/24"
Enter fullscreen mode Exit fullscreen mode

Objects inside a collection get matched by an identity field (id, name,
key, cidr_block, a few others) when every member carries one and it's
unique. So an edited security group rule reads as an edit:

ingress["web"].to_port  80 → 8080
Enter fullscreen mode Exit fullscreen mode

rather than one object leaving and a nearly identical one arriving.

Long values are trimmed around the difference, not from the start. Two
values sharing a 200-character prefix would otherwise print that prefix twice
and hide the part that changed. A string that's itself a JSON document — an IAM
policy, say — is shown as that document rather than as an escaped string.

Repeated failures get counted, not repeated. A removed provider
configuration produces one diagnostic per orphaned resource, which terraform
prints as forty paragraphs. The count is the news:

FAILED (1)
  ✗ infra/networking  ×38
      Error: Provider configuration not present: To work with module.peering-… (orphan)
Enter fullscreen mode Exit fullscreen mode

Same at stack level. One expired credential hits every unit; you get it once,
with the list:

FAILED (5)
  ✗ 5 units, same error
      envs/dev/a, envs/dev/b, envs/prod/a, +2 more
      Error: no valid credential sources found
Enter fullscreen mode Exit fullscreen mode

Folding never drops where the problem is — diagnostics sharing a message but
naming different lines list those lines.

Drift is a finding, not work

Nothing in drift changes when you apply. So it's counted rather than listed, and
a plan whose only findings are drift reports what terraform reports — no
changes:

SUMMARY  no changes
  3 resources drifted outside terraform, none of them addressed by this plan (--drift to list)
Enter fullscreen mode Exit fullscreen mode

--drift lists them, split by what the plan intends to do:

DRIFT — this plan puts it back (2)
DRIFT — this plan leaves it (1)
Enter fullscreen mode Exit fullscreen mode

The second is the one that bites — an attribute under ignore_changes, or a
resource the config no longer governs, stays drifted after the apply. Drift
never trips --fail-on or --detailed-exitcode, because those describe what an
apply will do.

tgsieve apply applies the plan you reviewed

Not a fresh plan made after you answered.

tgsieve apply --all
Enter fullscreen mode Exit fullscreen mode 
apply 9 changes across 5 units? [yes/no] yes
4 resources will be destroyed or replaced — type 'destroy' to confirm: destroy
Enter fullscreen mode Exit fullscreen mode

The second question only appears when something will be destroyed or replaced,
and it wants that word rather than another "yes" — those are the changes that
running the tool again won't undo. Outside a terminal it refuses rather than
assuming; --auto-approve is how you say you meant it in CI.

Review now, apply later:

tgsieve plan  --all --keep-plans ./plans --out-dir ./plans
tgsieve apply --all --plans ./plans
Enter fullscreen mode Exit fullscreen mode

And when the apply finishes — or stops — the report says what terraform
actually did, which after a failure is the one question the plan can't answer:

APPLY FAILED
  stopped after 15m45s — the report above is what was planned, not what landed
  ✗ Error: updating EKS Node Group (…) config: operation error EKS
  terraform changed 4 resources · 1 did not finish · slowest first
    ✗ terraform/live/ctrl/tests/eks module.eks…aws_eks_node_group.this — modifying, did not finish after 15m2s
    ✓ terraform/live/ctrl/tests/eks aws_security_group_rule.node created in 3s
  run tgsieve plan to see where things actually stand
Enter fullscreen mode Exit fullscreen mode

Living with a heavy stack

If your stack takes twenty minutes, you need more than a pretty report.

You can see what's happening. A window that updates in place instead of
scrolling:

  envs/prod/a  aws_db_instance.main     modifying… 1m12s
  envs/prod/b  aws_instance.web[3]      creating… 22s
  envs/dev/a   null_resource.deploy     done 4s
⠴ applying · 7/12 applied · 1m30s
Enter fullscreen mode Exit fullscreen mode

The denominator comes from terragrunt find, so it's known before the first
unit starts. Outside a terminal it collapses to a heartbeat line every 30
seconds, so CI logs still show liveness.

Ctrl-C behaves. It forwards the interrupt so terraform can release its state
locks, prints the report for whatever finished, lists the rest under NOT RUN,
and exits 130.

--resume picks up where an interrupted run stopped:

tgsieve plan --all --keep-plans ./plans          # 40 units, Ctrl-C at 31
tgsieve plan --all --keep-plans ./plans --resume # runs the missing 9
Enter fullscreen mode Exit fullscreen mode

Reusing a plan is only sound if the code hasn't moved under it, so a run records
the commit plus a fingerprint of uncommitted changes, and --resume refuses to
mix generations:

the plans in ./plans were made at 1cb275fd, the working tree is now at 4a91e0c2
  re-run without --resume to plan the stack fresh, or pass --force to mix generations
Enter fullscreen mode Exit fullscreen mode

That check also covers where each unit's code comes from. A remote module
pinned to a branch — or to a tag someone can move — reads identically before and
after the code it names changes, so each remote ref gets resolved to a commit
with git ls-remote (one call per distinct repo+ref, not per unit;
--no-resolve-refs for air-gapped runs).

--fast skips the refresh, which on a heavy stack is the single biggest
speed-up available. The summary says so every time, because a plan that never
looked at reality can report "no changes" for a stack that has drifted:

state was not refreshed: anything changed outside terraform is invisible here
Enter fullscreen mode Exit fullscreen mode

Plus: plan directories are locked while a run writes to them (--lock-wait 2m
for CI pipelines racing each other), and unit durations are remembered per
directory with a two-week TTL, so --timings stays meaningful across resumes.

In CI

--format decides the shape: tty (default), md, json or github.

Markdown for pull request comments — destructive changes stay open,
everything else folds into <details>, output is capped so GitHub doesn't
reject it, and every report starts with <!-- tgsieve --> so a bot can update
its own comment instead of adding a new one each run.

With Atlantis, a custom workflow:

workflows:
  tgsieve:
    plan:
      steps:
        - init
        - run: tgsieve plan --all --format md --fail-on high
    apply:
      steps:
        - run: tgsieve apply --all --auto-approve --format md
Enter fullscreen mode Exit fullscreen mode

--fail-on high turns the plan step red only when something is destroyed or
replaced — a pipeline can stop for a replacement without stopping for a new log
group.

JSON is a versioned document with its own types, not the internals of the
sieve:

tgsieve plan --all --format json | jq -r '.changes[] | select(.action=="replace") | .address'
Enter fullscreen mode Exit fullscreen mode

Sensitive values never appear in it — an attribute terraform marked sensitive is
reported as "sensitive": true with no before or after. A machine-readable
report is the easiest place for a secret to end up somewhere it shouldn't.

GitHub Actions annotations, so failures land on the diff rather than only in
the job log:

::error file=modules-vpcs.tf,line=72,title=infra/networking::Error: Unsupported attribute…
Enter fullscreen mode Exit fullscreen mode

And the exit codes distinguish the three ways a run can be unhappy:

code meaning
0 ran fine
1 tgsieve itself failed
2 changes survived the sieve (--detailed-exitcode, or --fail-on was met)
3 one or more units failed to plan
130 interrupted with Ctrl-C

No terragrunt? Still works

A root module big enough to be unreadable has the same problem as a stack, minus
the queue:

tgsieve plan  --engine terraform
tgsieve apply --engine terraform
Enter fullscreen mode Exit fullscreen mode

Same rules, same collapsing, same formats. The flags that only mean something
with a queue behind them say so rather than being quietly ignored:

--all needs terragrunt: the terraform engine plans one root module
Enter fullscreen mode Exit fullscreen mode

Try it 

brew install imcitius/tap/tgsieve
Enter fullscreen mode Exit fullscreen mode 
go install github.com/imcitius/tgsieve@latest
Enter fullscreen mode Exit fullscreen mode

Then, in a stack you already have:

tgsieve plan --all
Enter fullscreen mode Exit fullscreen mode

No config needed to start — with an empty .tgsieve.yaml, nothing is hidden and
you still get the nesting, the collapsing, the folded failures and the honest
summary. Add rules once you've seen which noise is your noise:

tgsieve init            # starter config at the project root
tgsieve rules           # what config is in effect, and from where
tgsieve plan --all --explain   # every hidden attribute and the rule that hid it
Enter fullscreen mode Exit fullscreen mode

It's MIT, written in Go, single binary, no daemon, no account.

👉 github.com/imcitius/tgsieve

I'd genuinely like to know what noise it doesn't catch on your stack — that's
the feedback that turns into the next preset. Issues, or just reply here.

Top comments (0)