Dozens of projects, one control plane. The modules were the easy part.
Pull up a chair, grab a cup of hot chai, and let’s talk straight for a minute.
Whenever someone sits down with me to ask about managing Terraform—which is just the tool we use to build cloud infrastructure through code instead of clicking buttons in a browser console—they always ask about modules. A module is simply a reusable bundle of Terraform code, like a shared template or function you write once so you don’t rewrite VPC networking or database configs forty times.
People ask: “Honey, how should I organize them? Should we put everything in a single massive Git repository, or split every module into its own repo? How generic and flexible should we make the inputs?”
Look, those are sensible questions. They sound very architect-level. But they are also completely wrong if you want to know what actually keeps you awake at night when your systems grow.
Across dozens of Google Cloud (GCP) projects, the code inside the modules was never what blew up in our faces. The modules were the easy part. The real battle was everything surrounding them: how you lock versions, how you isolate your state files (the digital ledger Terraform keeps to remember what real cloud machines belong to which line of code), how you catch drift when reality slips away from code, and how on earth you get humans to stop running terraform apply straight off their personal laptops.
Here is what actually matters, ranked by the scar tissue it leaves behind.
One control plane, dozens of projects. Every arrow below is a place the system around the modules can break.
1. Pin your module versions like your uptime depends on it (it does)
Let’s start with the quickest way to ruin an otherwise peaceful evening: pointing your Terraform code at the tip of a Git branch or writing module calls that just grab whatever happens to be “latest.”
Think of it like cooking for a banquet. If you order flour from a supplier by saying “send me whatever sack is closest to your warehouse door today,” one day they send you cement powder and your entire kitchen shuts down. You want the exact batch number stamped on the bag.
Notice the tiny difference between these two blocks of code below—just a few characters at the end of the URL, but they represent two completely different universes of safety:
# DON'T — this is a landmine
module "network" {
source = "git::https://<git-host>/<org>/terraform-modules.git//network"
}
# DO — boring, explicit, safe
module "network" {
source = "git::https://<git-host>/<org>/terraform-modules.git//network?ref=v2.4.1"
}
In the first block, Terraform fetches the latest commit off the default branch whenever someone runs an init. In the second block, the ?ref=v2.4.1 tag pins the code to a concrete, unshakeable release tag.
Unpinned modules mean someone else’s “quick fix” can roll out to every project on the next apply.
Imagine your teammate updates a default firewall rule or changes a subnet mask inside the shared networking module to fix an issue in their own development sandbox. They merge it to main thinking, “I’ll test this properly later.” Two hours later, you run an apply on a critical service. Because your module was unpinned, Terraform silently slurps that new change into your plan. The plan output is technically accurate, but it is 400 lines long. Nobody reads every single line of a giant plan diff on a busy afternoon. You approve it, and suddenly your production routing table just reshuffled itself.
Pin everything. Tag your Git releases with semantic versions. Upgrading a module in a project should always be a deliberate, conscious choice made one project at a time.
2. State: one per project per environment, no exceptions
When teams are small, people love getting clever with Terraform state. In plain English, the state file is Terraform’s map of the world. It maps your code declarations to the actual physical cloud resource IDs out in GCP or AWS. If you lose or mess up this file, Terraform doesn’t know what belongs to it anymore.
When folks start out, they try tricks: using built-in workspaces to juggle environments inside one giant file, or sharing a single state bucket across half the company because “these two projects talk to each other anyway.”
Do not do this. At scale, cleverness in state management is simply future incident post-mortems waiting to be written. The golden rule that actually survives contact with reality is dead simple:
One state file per project per environment. Full stop.
State isolation. When a state file corrupts or locks, the blast radius is exactly one project in one environment.
The term you hear SREs use here is “blast radius”—which is just a vivid way of asking: if this one thing explodes, how much of the surrounding neighborhood catches fire? If all your environments share one state file, a single bad deployment or corrupted state lock freezes the entire company. If each project and environment gets its own file, an error in staging only touches staging.
Here is how you enforce that isolation in Google Cloud Storage (GCS) using the backend configuration block:
terraform {
backend "gcs" {
bucket = "<tf-state-bucket>"
prefix = "<project-id>/<environment>/networking"
}
}
This backend block tells Terraform to store its state in a specific storage bucket, separated cleanly by project ID and environment under its own directory prefix. Clean boundaries make for peaceful sleep.
3. Drift detection is not optional — it’s the whole game
Here is the uncomfortable truth about infrastructure as code: code says what you wish was happening, while cloud consoles reflect what actually happened. The mismatch between the two is what we call “configuration drift.”
Here is how it happens every single time. It is 2:00 AM on a Sunday. An alert rings. A service cannot reach a database. An engineer on call with administrative access logs into the cloud web console, clicks into the firewall settings, adds an emergency IP rule, gets the traffic flowing, and saves the day. They fully intend to write a pull request on Monday morning to mirror that change in Terraform. But Monday morning brings three new urgent customer issues, a sprint planning meeting, and a broken build. The manual edit is forgotten.
This is not a character flaw. It is human nature under operational pressure. When the production house is on fire, nobody pauses to craft a beautiful Git commit.
Because you cannot stop humans from doing this with policy, you must catch it with automation:
Run scheduled dry-runs in CI: Set up your continuous integration runner (whether you use GitHub Actions, GitLab CI, or open-source tooling like OpenTofu runners) to execute
terraform planon a schedule—nightly at minimum—across every single project and environment. This command simply checks if reality matches code. Crucial rule: it reports the differences, but it never applies them automatically without human review.Treat drift as a standard alert: If a scheduled plan discovers unexpected changes, send a notification to the platform team just like you would for a failing health check. Drift is not an annoyance; it is an early warning indicator.
Normalize the backport without guilt: Establish a clear operational pact. Emergency console changes during an active incident are completely acceptable—provided that every manual fix creates an automatic ticket to bring that change back into code before the ticket closes. Zero shaming, just clean operational hygiene.
The drift loop: scheduled plans everywhere, reported centrally, backported deliberately.
Teams with scheduled drift detection catch small discrepancies when they are easy to reconcile. Teams without it only discover drift months later during an urgent deployment, right when Terraform cheerfully proposes to delete a load balancer or security rule that someone spent half the night hand-tuning six months ago.
4. Nobody runs apply from a laptop
This rule is purely cultural, and it is almost always the hardest one to enforce across an engineering team. Running terraform apply from a personal laptop invites every classic disaster at once:
First, there is zero centralized audit trail—nobody knows who applied what, with what flags, or when. Second, you end up with “it worked on my machine” applied to cloud infrastructure, where differences in local Terraform versions, provider plugins, or OS environments yield unpredictable plans. Third, if someone’s Wi-Fi drops halfway through applying a critical resource, Terraform can leave the remote state file locked, blocking every other team member until someone manually intervenes. And worst of all, it requires broad, write-level cloud credentials saved on local developer machines that can easily get misplaced, leaked, or exposed.
Same apply, two universes. Only one of them has an audit trail.
The standard solution is straightforward: every single infrastructure modification must execute inside an automated CI/CD pipeline, triggered strictly from the main Git branch, only after the plan output has been thoroughly inspected. The deployment pipeline is the sole entity granted write credentials to the production cloud environment. Engineers propose changes via pull requests, the automated system runs a plan and posts the diff directly as a comment for the team to inspect, a second engineer provides a formal review approval, and the pipeline executes the apply. No rogue terminals, no missing history.
5. Conventions beat cleverness
When you manage dozens of projects, the highest-leverage engineering decisions you will make are not flashy. They are mundane standards like consistent naming, folder structures, and concise documentation:
Naming: Stick to a strict <project>-<environment>-<resource> pattern across every resource, backed up by automated linter checks and peer reviews. When you are debugging an outage in the middle of the night and scanning logs or cloud resources, consistent naming saves you from guesswork.
Layout: Use an identical directory structure across every single repository. When a team member opens a networking repo or a database repo, the files should sit in the exact same spots. That predictability cuts developer onboarding time down from weeks to a couple of afternoons.
READMEs: Every shared module needs documentation covering four essential items: what inputs it expects, what outputs it produces, a working copy-paste code example, and—most importantly—what it intentionally does not do. Stating the boundaries upfront is what prevents a well-behaved, focused module from ballooning into an unmaintainable all-in-one framework over time.
None of these things look glamorous on a whiteboard. But over months and years, they compound into an infrastructure platform that simply works.
The takeaway
If your organization is scaling up its Terraform footprint, resist the urge to spend all your cycles designing deeply abstract, clever module hierarchies. Put 20% of your energy into writing clean modules, and put the remaining 80% into the boring, foundational operational practices around them: pin your release versions, isolate your state files, automate drift detection, run all applies through secure CI pipelines, and stick to boring conventions. The code inside the modules is the easy part. The operational discipline around them is the actual job.
Disclaimer
This post describes my personal experience with my own systems and accounts, in my own words. It is a first-person account of what I observed and what I do in my own work — not a statement of any company's intentions, policies, or internal decisions. Where I do not know something, the post says so.
I am not affiliated with, endorsed by, or speaking for any company, product, or vendor named in this post. All product and company names are trademarks of their respective owners and are mentioned only to describe the experience accurately.
Any error messages, documentation excerpts, or figures quoted here are reproduced from my own records or from the public sources linked in the post, believed accurate at the time of writing. Product behavior changes over time; verify against current official documentation before relying on any detail.
Nothing in this post is professional, financial, legal, or security advice. Examples are illustrative — validate in your own environment before running anything in production.
Corrections are welcome: if a company named here publishes an official statement, or a detail above is wrong, I will update this post and note the correction. The goal is an accurate public record of my experience, not a verdict on anyone.
© 2026 Honey Kiran. All rights reserved.
Honey Kiran · DevOps & Platform Engineering
Exploring the core platform patterns that keep large-scale cloud infrastructure resilient, maintainable, and predictable across monitoring, SRE, Terraform, and platform engineering.
Share this with the engineer on your team who still keeps production cloud keys on their laptop terminal.
Examples are illustrative — validate in your own environment before running in production.
Top comments (0)