Originally published on kuryzhev.cloud
Consider an illustrative scenario. A pull request adds one line to a security group resource, and the pipeline runs terraform plan. An LLM reviewer summarizes the diff as "minor tagging update," and a human approves it almost immediately. The actual plan output shows a forced replacement of the primary RDS instance, because a subnet group reference changed upstream. Nobody read past the AI summary. This is the failure mode teams walk into when they bolt an LLM reviewer onto Terraform plan changes and treat it as a stamp of trust, instead of as a component with its own failure surface.
Failure scenario
A platform team introduces an LLM reviewer that consumes the JSON output of terraform plan -out=tfplan && terraform show -json tfplan and posts a comment on the pull request. The comment reads well: bullet points, a risk rating and a plain-English summary. For a while it works fine on additive changes such as new S3 buckets, new IAM policies and extra tags.
Then a module update bumps a provider version, and the default value of an attribute that forces replacement changes. The human-readable plan shows -/+ resource "aws_db_instance" with a # forces replacement annotation. In the JSON, the same change appears only as an actions array of ["delete", "create"] plus replace_paths and action_reason fields. The LLM's summary mentions "instance configuration updated" without flagging replacement. The human reviewer approves, having learned to trust the AI summary after weeks of accurate output. The apply destroys and recreates a production database.
A second, quieter variant: the LLM reviewer is given a token budget that truncates large plans. Imagine a plan several thousand lines long for a multi-account VPC change, cut off partway through, with the destructive change sitting past the cut. The model reviews what it was given, honestly, and says nothing about what it never saw. Nothing in the pipeline output indicates truncation happened, so the missing risk is invisible to everyone downstream.
Why it happens
An LLM reviewer for Terraform plan changes is usually built to summarize text, not to reason reliably over structured infrastructure state. Terraform's JSON plan format encodes replacement, deletion and sensitive-value changes in specific fields:
-
actions, where replacement is["delete","create"]or["create","delete"], never a literal "replace"; -
replace_paths; -
action_reason; -
before_sensitiveandafter_sensitive.
A general-purpose prompt that just says "review this Terraform plan for risk" will not consistently surface these fields. The model's fluency creates false confidence. A clear, well-structured sentence about a risky change is only as good as its accuracy, and it reads exactly like an accurate one.
Token limits compound this. Large plans, especially from monorepos or wide blast-radius modules, can exceed context windows or budget-limited API calls. Truncation is silent unless the pipeline explicitly checks for it. Frontier LLMs and smaller local open-weight models both have this ceiling: the model doesn't know what it wasn't shown.
Watch out for teams treating "the LLM approved it" or "the LLM flagged it green" as equivalent to a policy check. An LLM reviewer is a heuristic summarizer, not a gate with guaranteed recall. It can miss a pattern it wasn't prompted to look for, especially a newly introduced one like a provider default change.
Prompt drift is the other quiet cause. As teams tweak the prompt to reduce false positives, such as alert fatigue on tagging changes, they can loosen the language enough that genuine risk categories get deprioritized. Replacement, deletion and IAM privilege escalation can slip down the list without anyone noticing the regression until a bad apply happens.
The fix (with code)
Treat the LLM as a second opinion layered on top of deterministic checks, never as the sole gate. Start with structured, non-AI detection of destructive actions directly from the JSON plan. Using jq against Terraform's documented JSON plan format is enough. Because replacements are always encoded as a delete paired with a create, matching on delete catches both deletions and replacements.
# Generate a plan and machine-readable JSON output
terraform plan -out=tfplan
terraform show -json tfplan > plan.json
# Count resources that will be deleted or replaced.
# Replacement appears as ["delete","create"] or ["create","delete"].
# "// []" avoids a jq error (and a fail-open gate) when there are no changes.
destructive=$(jq '
[(.resource_changes // [])[]
| select(.change.actions | index("delete"))]
| length
' plan.json) || { echo "PLAN_PARSE_FAILED"; exit 1; }
if [ "$destructive" -gt 0 ]; then
echo "DESTRUCTIVE_CHANGE_DETECTED: $destructive resource(s)"
exit 1
fi
This deterministic gate runs before the LLM ever sees the plan. A replacement or deletion therefore cannot slip through on the strength of a friendly summary, and a plan that fails to parse blocks the pipeline instead of passing it. Only after this check passes, or fails and receives an explicit human override, should the LLM reviewer generate its explanation for reviewers.
Feed the LLM a scoped extract of the plan rather than the raw JSON, and require it to return structured output you can validate programmatically instead of free text.
import json
def load_plan(path="plan.json"):
with open(path) as f:
return json.load(f)
def summarize_risk(plan):
risky = []
for c in plan.get("resource_changes", []):
change = c["change"]
actions = change["actions"]
# Replacement is encoded as delete+create, so "delete" covers it.
# "update" is included so in-place changes are also reviewed.
if "delete" in actions or "update" in actions:
risky.append({
"address": c["address"],
"actions": actions,
"replacement": "delete" in actions and "create" in actions,
"replace_paths": change.get("replace_paths", []),
"action_reason": c.get("action_reason"),
"touches_sensitive": bool(
change.get("before_sensitive") or change.get("after_sensitive")
),
"provider": c.get("provider_name"),
})
# This structured payload, not raw text, goes to the model.
# Required response schema: {verdict, cited_addresses[], reasons[], confidence}.
return risky
if __name__ == "__main__":
plan = load_plan()
risky_changes = summarize_risk(plan)
if risky_changes:
print(json.dumps(risky_changes, indent=2))
Route the structured risk list, not the entire plan, to a frontier LLM or to an internal service backed by Amazon Bedrock or the OpenAI API. If the list is still too large for the model's context, split it by resource address into multiple requests rather than truncating it. Require a schema-constrained response containing a verdict, cited resource addresses, reasons and a confidence score. Reject any response that doesn't parse against the schema, and treat a malformed response as "review failed," not "review passed." Log every plan/response pair for audit, since auditors or postmortems may later need to know exactly what the model saw.
Prevention checklist
Before relying on an LLM reviewer for Terraform plan changes in a merge gate, verify the following:
[ ] Deterministic check runs first: any delete action (including the
delete half of a replacement) blocks merge without human override,
independent of LLM output.
[ ] Plan JSON size is measured before sending to the model; oversized
plans are chunked by resource address, never silently truncated.
[ ] LLM output is schema-validated (verdict + cited resources), and a
malformed or missing response is treated as "block," not "pass."
[ ] Sensitive value changes (before_sensitive/after_sensitive fields)
are surfaced explicitly, not summarized away as "no visible change."
[ ] The exact prompt version is pinned and versioned in the repo, so a
prompt edit is reviewable like any other pipeline change.
[ ] A regular sample of approved plans is manually re-reviewed against
the LLM's verdict to catch silent accuracy drift.
[ ] IAM/policy-widening changes have a separate, stricter rule set,
since privilege escalation risk isn't always visible in plan diffs.
[ ] Reviewers are trained that the LLM summary is advisory; the plan
output linked in the PR is the source of truth, always.
None of this replaces standard Terraform hygiene such as remote state locking, mandatory terraform validate and module version pinning. It sits on top of it. For teams building this kind of guarded pipeline from scratch, the broader CI/CD and infrastructure automation patterns covered on kuryzhev.cloud are a reasonable starting point before layering AI review on.
Verify plan JSON field names against the HashiCorp documentation for your Terraform version, and check the format_version field in the output. The format is versioned, and newer releases may add fields that a reviewer tuned to an older schema will ignore or misread.
Top comments (0)