DEV Community: Amit Malhotra

Zero Trust Requires IAM Hygiene, Not Just Products

Amit Malhotra — Tue, 14 Apr 2026 15:04:45 +0000

Zero Trust Isn't a Product — It's What Happens When You Actually Review IAM

Most GCP organizations I assess have a zero trust problem they don't know about. They've configured VPC Service Controls. They've enabled BeyondCorp. They've checked the "zero trust" boxes on their security roadmap. But when I export their IAM bindings to BigQuery and run a simple query, I find service accounts with roles/editor granted two years ago that have never been reviewed.

Zero trust without IAM hygiene is security theater. The perimeter controls are there, but inside the perimeter, every service account has the keys to the kingdom.

The Problem Nobody Wants to Own

Least privilege is the goal. Everyone agrees on this. The problem is that nobody achieves it manually across a GCP org with dozens of projects and hundreds of service accounts.

Here's the pattern I see repeatedly in mid-market SaaS companies:

Initial platform setup happens fast — engineers grant roles/owner to service accounts because it works and they're under deadline pressure
Security reviews happen quarterly (if at all) and focus on project-level IAM, missing org-wide patterns
Nobody has a clear owner for IAM hygiene, so recommendations pile up indefinitely
SOC 2 auditors ask for evidence of periodic access reviews, and the team scrambles to produce manual spreadsheets

The fundamental issue isn't technical capability. GCP gives you everything you need to operationalize least privilege. The issue is that IAM governance requires a workflow, an owner, and a system of record. Most organizations have none of these.

IAM Recommender Exists — But Nobody Uses It Properly

IAM Recommender is one of the most underutilized tools in GCP. It automatically surfaces over-privileged bindings — roles granted that haven't been used in 90 days. It's doing the analysis work that would take a human weeks to do manually.

But here's what I've seen: teams enable IAM Recommender, look at the recommendations once, feel overwhelmed by the volume, and never act on them.

The recommendations pile up. Nothing changes. The audit comes around, and the team is in the same position they were in a year ago.

The missing piece is the analysis layer. IAM Recommender gives you individual recommendations per principal per resource. That's useful for tactical fixes, but it doesn't give you the strategic view. You can't see patterns across your org. You can't prioritize by risk. You can't track remediation progress over time.

This is where BigQuery changes the game.

Operationalizing Zero Trust with BigQuery

Exporting IAM Recommender data to BigQuery lets you run org-wide analysis at scale. Instead of reviewing recommendations one by one in the console, you can query your entire IAM posture programmatically.

Start with Cloud Asset Inventory to export IAM bindings:

gcloud asset export \
  --organization=ORG_ID \
  --billing-project=PROJECT_ID \
  --asset-types="iam.googleapis.com/ServiceAccount" \
  --output-bigquery-table projects/PROJECT/datasets/DATASET/tables/iam_export

Then query for the highest-risk patterns — service accounts with roles/editor or roles/owner:

SELECT
  resource.name,
  iam_policy.bindings.role,
  iam_policy.bindings.members
FROM `project.dataset.iam_export`
WHERE iam_policy.bindings.role IN ('roles/editor','roles/owner')

In one SaaS company I worked with, this query revealed 47 service accounts with roles/editor at the project level. Fifteen of those service accounts had additional roles — some with 15+ unused permissions going back two years. The platform team had no idea.

For recommendations specifically, use the Recommender API:

gcloud recommender recommendations list \
  --recommender=google.iam.policy.Recommender \
  --location=global

You can also integrate IAM Recommender findings with Security Command Center. Recommendations surface as findings with the google.iam.policy.Insight finding type. Route these to your ticketing system, and you've got an automated workflow that didn't exist before.

What Changes When You Have the Data

Once you have IAM analysis in BigQuery, several things become possible:

Risk prioritization. Not all over-privileged bindings are equal. A service account with roles/owner on your production data project is more urgent than one with roles/editor on a sandbox project. BigQuery lets you join IAM data with resource metadata to prioritize by blast radius.

Remediation tracking. Run the same query weekly. Track the count of high-risk bindings over time. Show the trend line to auditors. This is the evidence of continuous improvement that SOC 2 controls require.

Ownership visibility. BigQuery analysis often reveals that nobody knows who created certain service accounts or why they exist. This visibility forces the conversation about IAM ownership that most orgs avoid.

The Lifecycle Operations stage of the SCALE Framework is where most teams fall short. They have security controls in place, but no ongoing governance process. BigQuery + IAM Recommender gives you the operational layer that makes governance sustainable.

Trade-Offs You Need to Understand

This approach isn't without complexity.

90-day usage window limitations. IAM Recommender looks at the last 90 days of activity. If you have seasonal workloads or jobs that run quarterly, they'll get flagged as unused. Review recommendations before auto-remediating. I've seen teams accidentally revoke permissions from their disaster recovery service accounts because those accounts only get used during DR tests.

Custom role maintenance burden. The proper remediation for over-privileged bindings is often a custom role scoped to actual API usage. But custom roles require maintenance. When GCP releases new APIs, custom roles don't automatically get new permissions. Someone has to own the role lifecycle, or you'll break workloads when GCP updates services.

Point-in-time exports. A single BigQuery export gives you a snapshot. For continuous monitoring, set up scheduled exports via Cloud Asset Inventory feeds. This adds infrastructure to maintain, but it's the only way to make IAM governance truly continuous.

The Question You Need to Answer

Zero trust is an architecture principle, not a product you buy. IAM Recommender gives you the data. BigQuery gives you the analysis layer. The tools exist.

What's missing in most organizations is the remediation workflow and ownership. If nobody owns IAM hygiene, the recommendations pile up and nothing changes. You'll have all the visibility in the world and no improvement to show for it.

The question isn't whether to implement this pattern. The question is: who in your organization owns IAM governance, and what happens when they find 200 over-privileged service accounts?

What's the oldest unused role binding you've found in your GCP org? I've seen some that predate the company's SOC 2 certification by years.

Amit Malhotra, Principal GCP Architect, Buoyant Cloud Inc

Work with a GCP specialist — book a free discovery call

Work with a GCP specialist — book a free discovery call → https://buoyantcloudtech.com

Zero Trust Requires IAM Hygiene, Not Just Products

Amit Malhotra — Tue, 07 Apr 2026 14:58:34 +0000

Zero Trust Isn't a Product — It's What Happens When You Actually Review IAM

Zero trust without IAM hygiene is security theater. The perimeter controls are there, but inside the perimeter, every service account has the keys to the kingdom.

The Problem Nobody Wants to Own

Least privilege is the goal. Everyone agrees on this. The problem is that nobody achieves it manually across a GCP org with dozens of projects and hundreds of service accounts.

Here's the pattern I see repeatedly in mid-market SaaS companies:

Initial platform setup happens fast — engineers grant roles/owner to service accounts because it works and they're under deadline pressure
Security reviews happen quarterly (if at all) and focus on project-level IAM, missing org-wide patterns
Nobody has a clear owner for IAM hygiene, so recommendations pile up indefinitely
SOC 2 auditors ask for evidence of periodic access reviews, and the team scrambles to produce manual spreadsheets

IAM Recommender Exists — But Nobody Uses It Properly

But here's what I've seen: teams enable IAM Recommender, look at the recommendations once, feel overwhelmed by the volume, and never act on them.

The recommendations pile up. Nothing changes. The audit comes around, and the team is in the same position they were in a year ago.

This is where BigQuery changes the game.

Operationalizing Zero Trust with BigQuery

Start with Cloud Asset Inventory to export IAM bindings:

gcloud asset export \
  --organization=ORG_ID \
  --billing-project=PROJECT_ID \
  --asset-types="iam.googleapis.com/ServiceAccount" \
  --output-bigquery-table projects/PROJECT/datasets/DATASET/tables/iam_export

Then query for the highest-risk patterns — service accounts with roles/editor or roles/owner:

SELECT
  resource.name,
  iam_policy.bindings.role,
  iam_policy.bindings.members
FROM `project.dataset.iam_export`
WHERE iam_policy.bindings.role IN ('roles/editor','roles/owner')

For recommendations specifically, use the Recommender API:

gcloud recommender recommendations list \
  --recommender=google.iam.policy.Recommender \
  --location=global

What Changes When You Have the Data

Once you have IAM analysis in BigQuery, several things become possible:

Trade-Offs You Need to Understand

This approach isn't without complexity.

The Question You Need to Answer

Zero trust is an architecture principle, not a product you buy. IAM Recommender gives you the data. BigQuery gives you the analysis layer. The tools exist.

The question isn't whether to implement this pattern. The question is: who in your organization owns IAM governance, and what happens when they find 200 over-privileged service accounts?

What's the oldest unused role binding you've found in your GCP org? I've seen some that predate the company's SOC 2 certification by years.

Amit Malhotra, Principal GCP Architect, Buoyant Cloud Inc

Work with a GCP specialist — book a free discovery call

Work with a GCP specialist — book a free discovery call → https://buoyantcloudtech.com

Zero Trust Requires IAM Hygiene, Not Just Products

Amit Malhotra — Tue, 31 Mar 2026 14:53:24 +0000

Zero Trust Isn't a Product — It's What Happens When You Actually Review IAM

Zero trust without IAM hygiene is security theater. The perimeter controls are there, but inside the perimeter, every service account has the keys to the kingdom.

The Problem Nobody Wants to Own

Least privilege is the goal. Everyone agrees on this. The problem is that nobody achieves it manually across a GCP org with dozens of projects and hundreds of service accounts.

Here's the pattern I see repeatedly in mid-market SaaS companies:

Initial platform setup happens fast — engineers grant roles/owner to service accounts because it works and they're under deadline pressure
Security reviews happen quarterly (if at all) and focus on project-level IAM, missing org-wide patterns
Nobody has a clear owner for IAM hygiene, so recommendations pile up indefinitely
SOC 2 auditors ask for evidence of periodic access reviews, and the team scrambles to produce manual spreadsheets

IAM Recommender Exists — But Nobody Uses It Properly

But here's what I've seen: teams enable IAM Recommender, look at the recommendations once, feel overwhelmed by the volume, and never act on them.

The recommendations pile up. Nothing changes. The audit comes around, and the team is in the same position they were in a year ago.

This is where BigQuery changes the game.

Operationalizing Zero Trust with BigQuery

Start with Cloud Asset Inventory to export IAM bindings:

gcloud asset export \
  --organization=ORG_ID \
  --billing-project=PROJECT_ID \
  --asset-types="iam.googleapis.com/ServiceAccount" \
  --output-bigquery-table projects/PROJECT/datasets/DATASET/tables/iam_export

Then query for the highest-risk patterns — service accounts with roles/editor or roles/owner:

SELECT
  resource.name,
  iam_policy.bindings.role,
  iam_policy.bindings.members
FROM `project.dataset.iam_export`
WHERE iam_policy.bindings.role IN ('roles/editor','roles/owner')

For recommendations specifically, use the Recommender API:

gcloud recommender recommendations list \
  --recommender=google.iam.policy.Recommender \
  --location=global

What Changes When You Have the Data

Once you have IAM analysis in BigQuery, several things become possible:

Trade-Offs You Need to Understand

This approach isn't without complexity.

The Question You Need to Answer

Zero trust is an architecture principle, not a product you buy. IAM Recommender gives you the data. BigQuery gives you the analysis layer. The tools exist.

The question isn't whether to implement this pattern. The question is: who in your organization owns IAM governance, and what happens when they find 200 over-privileged service accounts?

What's the oldest unused role binding you've found in your GCP org? I've seen some that predate the company's SOC 2 certification by years.

Amit Malhotra, Principal GCP Architect, Buoyant Cloud Inc

Work with a GCP specialist — book a free discovery call

Work with a GCP specialist — book a free discovery call → https://buoyantcloudtech.com

Static Service Account Keys: Your Biggest GCP Identity Risk

Amit Malhotra — Tue, 24 Mar 2026 14:49:12 +0000

Static Service Account Keys Are Still Your Biggest GCP Identity Risk

Most GCP environments I audit have the same problem hiding in plain sight. Not misconfigured firewall rules. Not overly permissive IAM roles. Service account keys.

I find them in GitHub repos, in CI/CD environment variables, stored on developer laptops, committed to private repos that "nobody external can access." The teams running these environments aren't careless. They're experienced engineers who set up keys years ago when it was the standard approach, and nobody has had the bandwidth to migrate.

That key sitting in your Jenkins server is a ticking breach. And unlike a compromised password, a compromised GCP key doesn't trigger an account lockout after failed attempts. It just works — silently, indefinitely — until you notice the billing spike or the security incident.

The $450k Weekend

One team I worked with learned this the hard way. A service account key leaked through a public GitHub commit. The commit was reverted within hours, but the key was already harvested by automated scrapers. Over a single weekend, attackers spun up Cloud Run instances across every available region, running crypto mining workloads.

The bill: $450,000.

GCP support eventually provided credits, but the incident consumed weeks of engineering time, triggered their SOC 2 auditor's attention, and forced an emergency security review across their entire infrastructure.

The key had been valid for three years. Nobody remembered creating it.

What Most Teams Get Wrong

The solution to this problem has existed for years: Workload Identity Federation. External identities — GitHub Actions runners, GitLab CI, even AWS workloads — can exchange OIDC tokens for short-lived GCP credentials. No keys required.

For GKE workloads, Workload Identity lets Kubernetes Service Accounts impersonate GCP Service Accounts without any credentials stored in the cluster.

These aren't new features. They're production-ready and well-documented. So why do I still find keys everywhere?

Because teams implement one piece without completing the migration.

I see this pattern constantly:

WIF configured for GitHub Actions, but old keys left active "just in case the new approach breaks"
Workload Identity enabled on GKE, but legacy deployments still mounting key files as secrets
Org policy blocking key creation, but dozens of existing keys still valid and in use

The partial migration is almost worse than no migration. Your audit trail shows both authentication methods being used. Your security team can't tell which is legitimate. Your attackers now have two paths into your systems.

The Identity Pattern That Actually Works

Eliminating keys requires two components working together: Workload Identity Federation and Service Account Impersonation.

WIF handles machine-to-machine authentication. Your GitHub Actions workflow authenticates to GCP without storing any secrets:

- uses: google-github-actions/auth@v2
  with:
    workload_identity_provider: projects/123456/locations/global/workloadIdentityPools/github-pool/providers/github-provider
    service_account: deploy-sa@project.iam.gserviceaccount.com

No key to rotate. No secret to leak. The token expires automatically.

For GKE, the Kubernetes Service Account annotation binds to a GCP Service Account:

gcloud iam service-accounts add-iam-policy-binding deploy-sa@project.iam.gserviceaccount.com \
  --role roles/iam.workloadIdentityUser \
  --member "serviceAccount:project.svc.id.goog[production/app-ksa]"

Service Account Impersonation handles the human side. Instead of developers holding permanent credentials to a powerful service account, they impersonate a scoped service account on demand:

gcloud config set auth/impersonate_service_account deploy-sa@project.iam.gserviceaccount.com

The developer's identity is still the audit principal. You can see exactly who impersonated which account, when, and what they did. Compare that to five engineers sharing the same downloaded key file — your audit logs just show the service account, with no way to trace the actual human.

The Org Policy That Creates Friction

Once you're confident your workloads don't need keys, enforce it:

constraints/iam.disableServiceAccountKeyCreation

This org policy prevents anyone from generating new keys. I've seen it implemented successfully — and I've seen it create chaos.

The chaos happens when you enable the policy before educating your engineering team. Developers who don't know about WIF or gcloud auth application-default login suddenly can't authenticate their local development environments. They file urgent tickets. They complain about "security blocking progress." Some creative ones figure out workarounds that are worse than the original keys.

The migration order matters. Document the new authentication patterns. Train your developers. Set up WIF for CI/CD. Verify that no active workloads depend on keys. Then enable the org policy.

This sequence aligns with the Security by Design phase of our SCALE framework — identity architecture has to be right before you build automation on top of it.

The Trade-offs Nobody Mentions

WIF and impersonation aren't without friction.

Local development gets more complex. With keys, developers could just set GOOGLE_APPLICATION_CREDENTIALS and move on. With WIF, you need gcloud auth application-default login workflows documented and understood. Some developers will resist this. Your platform team needs to make the secure path the easy path.

Audit configuration has to be correct. Impersonation creates cleaner audit trails, but only if you're capturing the right logs. sts.googleapis.com events need to be in your Cloud Audit Logs configuration. I've seen teams implement impersonation and then realize months later that they weren't logging the token exchanges.

Cross-project impersonation gets complicated fast. A service account in Project A impersonating a service account in Project B that accesses resources in Project C creates a chain that's hard to audit and easy to misconfigure. Keep impersonation chains to one hop maximum.

What This Means for SOC 2

Every SOC 2 audit I've supported in the last three years has flagged service account keys. The auditors aren't wrong — long-lived credentials with no rotation policy and unclear ownership are a control gap.

The finding usually reads something like: "Service account keys exist without defined rotation schedules or ownership assignment."

You can write a policy that says keys must be rotated every 90 days. You can assign ownership in a spreadsheet. You can build automation to rotate keys. Or you can eliminate keys entirely and remove the finding at its root.

Eliminating keys is not optional for regulated SaaS. The migration path from keys to WIF is well-defined — the blocker is usually organizational, not technical. Someone has to own the project, inventory the existing keys, map them to workloads, and execute the migration without breaking production.

That's the work. It's not glamorous. It doesn't involve new tools or exciting architecture diagrams. But it's the single highest-impact security improvement most GCP environments can make today.

If identity boundaries are wrong, everything built on top of them inherits the risk.

Author: Amit Malhotra, Principal GCP Architect, Buoyant Cloud Inc

Work with a GCP specialist — book a free discovery call

Work with a GCP specialist — book a free discovery call → https://buoyantcloudtech.com

VPC Service Controls Private IP Gap: A Security Risk

Amit Malhotra — Tue, 17 Mar 2026 14:47:16 +0000

VPC Service Controls Without Private IP Coverage Is Security Theater

Most GCP teams I work with have VPC Service Controls enabled. They check the compliance box, show auditors the perimeter configuration, and move on. What they don't realize is that their internal services can still exfiltrate data to external projects without triggering a single alert.

The gap isn't in VPC-SC itself — it's in how teams deploy it. Private IP support in VPC-SC perimeters has been available for a while now, but I'd estimate fewer than 20% of the SaaS platforms I audit have actually implemented it. The rest have a perimeter that looks solid on paper but leaves the most common exfiltration path wide open.

The Exfiltration Path Nobody Talks About

VPC Service Controls were designed to prevent data from leaving your GCP organization through managed services like BigQuery, Cloud Storage, and Secret Manager. The original implementation worked well for public internet traffic — if someone tried to copy data from your BigQuery dataset to an external project over the public API, VPC-SC blocked it.

But traffic originating from private IP ranges inside your VPC? That wasn't covered.

Think about what that means in practice. An attacker compromises a service account on a GKE workload running in your private network. They have access to BigQuery through that identity. With VPC-SC but without private IP coverage, they can query your datasets and write the results to an external project they control — all from inside your "protected" perimeter.

I've seen this exact scenario during penetration tests. The security team was confident their VPC-SC perimeter would catch any data exfiltration attempt. It didn't. The test showed data flowing out through internal services that the perimeter didn't inspect.

Why This Gap Persists

Three patterns explain why most teams haven't closed this gap:

Dry-run mode paralysis. VPC-SC is notoriously difficult to enforce without breaking production services. The safe approach is to run in dry-run mode first, watch the logs, and then switch to enforced. I've seen teams run in dry-run mode for six months or longer. At that point, dry-run becomes the permanent state — and dry-run mode doesn't actually block anything. It just logs what would have been blocked. That's monitoring, not protection.

Incomplete perimeter design. Teams enable VPC-SC for BigQuery and Cloud Storage but skip Secret Manager, Cloud SQL Admin API, or other services that handle sensitive data. Attackers don't care which service holds your data — they'll take whatever path is open.

Misconfigured ingress and egress rules. Once private IP support is enabled, you need explicit ingress rules allowing legitimate internal traffic. Most teams either make these rules too broad (defeating the purpose) or too narrow (breaking production). The operational burden pushes teams toward permissive configurations or abandoning the feature entirely.

The Architecture That Actually Works

In my experience, closing the data exfiltration gap requires three components working together:

VPC-SC with private IP coverage. Configure your perimeter to inspect traffic from internal CIDR ranges, not just public internet traffic. This is the foundation.

Access Context Manager access levels tied to network origin and identity. Don't just allow traffic from a private IP range — require that traffic to also come from a specific service account. Defense in depth means an attacker needs to compromise both the network position and the identity.

Ingress rules scoped to specific services and methods. If your data pipeline only needs to read from BigQuery, don't grant write access through the perimeter. Principle of least privilege applies to network perimeters just like it applies to IAM.

Here's what a properly scoped ingress rule looks like in practice:

ingressPolicies:
  - ingressFrom:
      sources:
        - resource: "projects/YOUR_PROJECT"
      identities:
        - serviceAccount:data-pipeline@project.iam.gserviceaccount.com
    ingressTo:
      operations:
        - serviceName: bigquery.googleapis.com
          methodSelectors:
            - method: "google.cloud.bigquery.v2.JobService.Query"
      resources:
        - "projects/YOUR_PROJECT/datasets/production_data"

This rule allows one service account to run queries against one dataset. An attacker who compromises a different service account — or the same service account trying to write data externally — gets blocked.

The Terraform resource for managing this is google_access_context_manager_service_perimeter. I'd recommend managing all perimeter configuration through infrastructure as code. Manual console changes to VPC-SC perimeters are a recipe for configuration drift and broken production services.

What This Means for SOC 2 and Audit Readiness

During SOC 2 audits, I've had auditors ask specifically about data exfiltration controls. "Show me how you prevent an insider or compromised credential from copying data outside your organization."

VPC-SC without private IP coverage doesn't answer that question. You can show them the perimeter configuration, but if private IP traffic isn't covered, you have a control gap. Auditors who understand GCP will catch it. Auditors who don't will accept the checkbox — until you have an incident and the forensics reveal the gap.

This is where the security-by-design principle from the SCALE framework matters most. If you build your perimeter architecture correctly from the start, SOC 2 evidence collection is straightforward. If you retrofit private IP coverage onto an existing perimeter, you're doing the work twice and risking production outages during the transition.

The Trade-Offs Are Real

I'm not going to pretend VPC-SC with full private IP coverage is easy to operate. It adds friction to every new service deployment. Your platform team will field tickets from developers asking why their new Cloud Function can't reach BigQuery. The answer will be "because you didn't add it to the perimeter ingress rules," and that will slow down their sprint.

The CIDR planning requirements are also non-trivial. If you have overlapping IP ranges across projects — common in organizations that grew without central network governance — you'll hit routing issues that are painful to debug.

And not all GCP services support VPC-SC yet. Before designing your perimeter architecture, check the supported services list. Building a perimeter around services that don't support it creates gaps you can't close with configuration.

The Enforcement Question

If your VPC-SC perimeter has been in dry-run mode for more than a month, you need to ask yourself an honest question: is it ever going to be enforced?

Dry-run mode is valuable for the first two weeks. You watch the logs, identify legitimate traffic that would be blocked, and adjust your ingress and egress rules. After that, you either enforce the perimeter or admit that you're not actually protecting anything.

I've worked with teams who ran dry-run mode for eight months because they were afraid of breaking production. During that time, they had zero data exfiltration protection. The perimeter existed on paper. In practice, it was monitoring, not security.

Data exfiltration is the top concern in every regulated SaaS environment I work with. Your customers trust you with their data. VPC-SC with private IP support is one of the few controls that actually prevents data from leaving your organization through GCP's managed services.

If you've been putting off this work, the gap is still open. What's your plan to close it?

Work with a GCP specialist — book a free discovery call.

Amit Malhotra

Principal GCP Architect, Buoyant Cloud Inc

Work with a GCP specialist — book a free discovery call → https://buoyantcloudtech.com

GKE Security: Why Monitoring Isn't Enough for Compliance

Amit Malhotra — Tue, 10 Mar 2026 14:37:24 +0000

Your GKE Cluster Passed Security Review — And It's Still Not Audit-Ready

Most GKE clusters I audit look secure on paper. Workload Identity enabled. Private cluster networking. RBAC configured. The security checklist is complete, the platform team feels confident, and then the SOC 2 auditor asks a simple question: "Show me the control that prevents someone from deploying a cluster without these settings."

That's when the conversation gets uncomfortable.

The Gap Between Monitoring and Prevention

Here's the pattern I see repeatedly across SaaS companies running production workloads on GKE: teams implement security controls inside their clusters but leave the provisioning layer wide open.

Security Command Center is enabled. GKE Security Posture dashboard shows green. The team has even run the CIS GKE Benchmark manually and fixed the findings. Everything looks good.

But there's no preventive control at the organization level. Any engineer with the right IAM permissions can spin up a new cluster tomorrow with default settings — no Workload Identity, no Shielded Nodes, client certificates enabled. That cluster will appear in SCC findings eventually, but by then it's running production traffic.

The business risk here isn't theoretical. I've watched teams scramble during SOC 2 preparation when auditors ask for evidence that non-compliant infrastructure cannot be provisioned. "We monitor for drift" is not the same answer as "we prevent it at the platform layer."

What Most Teams Get Wrong About GKE Security

The CIS GKE Benchmark exists. GCP has native tooling to enforce it. Security Command Center can surface violations in real time. But teams implement these pieces in isolation rather than as a layered enforcement system.

In my experience working with B2B SaaS companies preparing for audits, the breakdown usually looks like this:

Detection without prevention. SCC is enabled, but findings pile up with no remediation workflow. The dashboard becomes noise. Nobody reviews it weekly because there's no escalation path.

Benchmark compliance as a point-in-time exercise. Teams run CIS scans before an audit, fix the violations, and move on. Six months later, a new cluster gets provisioned with the same issues because nothing prevents it.

Security posture without organizational enforcement. The GKE Security Posture dashboard now integrates beautifully with SCC to surface OS vulnerabilities, workload misconfigurations, and exposed secrets per workload. Most teams I work with don't know this integration exists — and even fewer use Organization Policies to enforce the same controls preventively.

The result is a cluster that looks secure but fails the audit question that actually matters: "What prevents this from happening in the first place?"

Org Policies Are the Missing Layer

Custom Organization Policies are where GKE security moves from reactive to preventive. Instead of detecting a non-compliant cluster after deployment, you block the creation of that cluster before it happens.

This is the Security by Design principle in the SCALE framework — your controls should prevent misconfiguration at provisioning time, not just detect it afterward.

Here's a concrete example. To enforce Workload Identity across your entire organization:

constraint: constraints/container.requireWorkloadIdentity

Apply this at the organization or folder level, and any gcloud container clusters create command that doesn't include Workload Identity configuration fails immediately. No cluster gets provisioned. No SCC finding to remediate later.

The same pattern applies to Shielded Nodes, client certificate issuance, and other CIS Benchmark controls. In Terraform, this enforcement happens before the cluster exists:

resource "google_container_cluster" "main" {
  workload_identity_config {
    workload_pool = "${var.project_id}.svc.id.goog"
  }
  enable_shielded_nodes = true
  master_auth {
    client_certificate_config {
      issue_client_certificate = false
    }
  }
}

When Org Policies are in place, Terraform plans that violate the policy fail during terraform plan — not after the cluster is running.

The SCC Integration Most Teams Miss

GKE Security Posture dashboard now integrates directly with Security Command Center. This surfaces top threats per workload in a unified view — OS vulnerabilities from base images, workload misconfigurations, secrets accidentally committed to pods.

I've seen teams enable SCC and enable GKE Security Posture as separate activities, never realizing they feed the same dashboard. The integration matters because it gives you one place to track both infrastructure-level compliance (Org Policies) and workload-level threats (runtime security findings).

If you're using SCC Enterprise tier, those findings also feed into Chronicle for SIEM correlation. That's useful for threat detection, but for most SaaS companies, the Standard tier's vulnerability findings are the priority for SOC 2.

The Trade-offs Are Real

Org Policies are blunt instruments. They enforce at the organization or folder level, which means a policy that makes sense for production clusters might block legitimate experimentation in development environments.

The solution is folder-level scoping. Put production projects under a folder with strict enforcement. Put sandbox and development projects under a different folder with relaxed policies. This gives you preventive controls where they matter without blocking engineers from learning.

SCC tier matters more than most teams realize. Standard tier gives you vulnerability findings — container image CVEs, misconfigurations, exposed secrets. Enterprise tier adds threat detection — suspicious network activity, potential lateral movement, compromise indicators. Most SaaS companies preparing for SOC 2 can start with Standard tier and upgrade when threat intelligence becomes a priority.

Retroactive enforcement is painful. If you have existing GKE clusters that violate the Org Policies you want to enforce, enabling those policies doesn't fix the existing clusters — it just blocks new non-compliant ones. You need a remediation plan for existing infrastructure, and that means scheduled maintenance windows.

What Auditors Actually Ask For

SOC 2 auditors increasingly ask for evidence of preventive controls, not just detective ones. This is the shift that catches teams off guard.

"We monitor for this" is no longer sufficient when the follow-up question is "What stops an engineer from bypassing that monitoring?"

Org Policies enforcing CIS Benchmark controls is the difference between those two answers. It's the evidence that your security posture is structural, not procedural. When an auditor asks how you ensure all GKE clusters use Workload Identity, you show them the constraint that blocks any cluster creation without it.

If you're heading into an audit with GKE clusters, this is the first place to look. Not because the technical implementation is complex — it isn't — but because it changes the nature of your control evidence from "we detect and respond" to "we prevent."

That distinction matters more than most teams realize until they're sitting across from an auditor explaining why a finding from six months ago is still open.

About the Author

Amit Malhotra is Principal GCP Architect at Buoyant Cloud Inc, where he helps B2B SaaS companies design audit-ready GKE platforms and implement the SCALE framework for cloud infrastructure.

Work with a GCP specialist — book a free discovery call

Work with a GCP specialist — book a free discovery call → https://buoyantcloudtech.com

Why Cloud Platforms Fail: It's the Sequence, Not the Tools

Amit Malhotra — Wed, 04 Mar 2026 00:40:26 +0000

Why Your Cloud Platform Keeps Breaking — It's Not the Tools, It's the Sequence

Most cloud failures I get called in to fix weren't caused by picking the wrong technology. They were caused by doing the right things in the wrong order.

I've spent years designing and securing GCP infrastructure for enterprises like RBC, Tangerine Bank, Telus Health, and Loblaws — plus dozens of high-growth B2B SaaS companies where security and speed both had to work at the same time. The pattern is consistent: platforms don't collapse because someone chose Cloud Run over GKE, or PostgreSQL over Spanner. They collapse because security got bolted on after architecture decisions were already locked in. Because infrastructure got provisioned manually "just this once" and stayed that way. Because scaling was something to figure out later.

Later always arrives faster than you expect.

The Five Problems Nobody Talks About Until the Audit

Here's what I keep seeing across engagements, regardless of company size or industry:

Operational complexity growing faster than the team. Every new service adds overhead nobody planned for. Your platform team that comfortably managed three services is now drowning in twelve, and the cognitive load has become unsustainable.

Environment drift. Dev, Staging, and Production are configured differently in ways nobody fully documented. Bugs appear only in prod. Deployments that passed every test still fail in ways that take days to diagnose.

Security gaps discovered during audits — not designed out from the start. SOC 2 Type II with Drata sounds straightforward until the auditor asks why your service accounts have project-wide editor permissions, or why your VPC has no network segmentation.

Scaling surprises. Platforms that work perfectly at 10,000 users fall over at 100,000. The architecture wasn't wrong — it just wasn't designed for what came next.

Cloud costs nobody can fully explain. Spend growing 40% quarter over quarter, but revenue only growing 15%. Finance asks what's driving it. The honest answer is "we're not sure."

These aren't tool problems. They're sequence problems.

My Take: Order of Operations Matters More Than Technology Choice

In my experience, the difference between platforms that scale gracefully and platforms that require emergency redesigns every 18 months comes down to one thing: whether the foundational decisions were made in the right order.

This is why I built the SCALE Framework — not as a product, but as a diagnostic lens and design sequence that addresses the five failure modes I kept encountering.

Security by Design (S) has to come first. Not because security is more important than everything else, but because security decisions made late are always weaker than security decisions made early. By the time most teams think seriously about IAM models, network segmentation, or workload identity, the architecture has already made those decisions for them — usually badly. Retrofitting Zero-Trust into a platform built on convenience-first permissions is painful, expensive, and never quite complete.

Cloud-Native Architecture (C) comes next, because how you structure workloads determines what's even possible for automation, scaling, and cost management later. Cloud-native doesn't mean "runs on cloud." It means the platform takes advantage of how cloud infrastructure actually works — managed services over self-managed VMs, containers over monoliths, regional redundancy over single points of failure.

Automation and Infrastructure as Code (A) locks in consistency. I've seen teams with brilliant architecture and solid security still suffer from environment drift because infrastructure was provisioned manually. If you can't reproduce your environment from code, you don't have infrastructure — you have a snowflake. Terraform-driven provisioning, where every environment is deployed from the same codebase, reviewed like code, deployed without manual steps.

Lifecycle Operations (L) — what most people call DevSecOps — treats deployment as part of development, not something that happens after. Security checks, automated testing, and policy validation run in the pipeline before anything reaches production. This is where the earlier decisions pay off: secure-by-design architecture, consistent infrastructure, automated deployments combine into a release process that's routine instead of risky.

Elastic Scalability and Efficiency (E) closes the loop. Scalability isn't just a technical requirement — it's a financial one. A platform that scales technically but doubles your cloud bill every time you grow 20% isn't a success. Dynamic scaling, right-sized resources, cost visibility tooling — so the team always knows what they're spending and why.

Why All Five Pillars Matter Together

Each pillar reinforces the others. A weakness in any one creates pressure on the rest.

Security gaps usually trace back to missing automation — manual processes create inconsistency, inconsistency creates gaps. Cost problems usually trace back to missing scalability design — resources provisioned for peak load running 24/7 because nobody built the auto-scaling logic. Operational overhead usually traces back to missing cloud-native architecture — teams managing infrastructure that should be managed services.

This is why I address all five in every engagement, even when a client comes to me with just one problem. The presenting issue is rarely the root cause.

What This Looks Like in Practice

A client came to me last year after a failed SOC 2 audit. The immediate problem was IAM — service accounts with too-broad permissions, no workload identity, manual access management. But the real problem was that their infrastructure had grown organically without automation. Every fix required touching multiple environments by hand. Implementing least-privilege IAM without infrastructure as code would have taken months and created more drift.

We started with security architecture, yes — but we implemented it through Terraform modules that standardized IAM across all environments simultaneously. Security improvement and automation improvement happened together, because they had to.

Six months later, they passed SOC 2 Type II with no findings. More importantly, their deployment frequency increased 3x because the team trusted the pipeline. That's what the right sequence produces.

Trade-Offs and Honest Limitations

SCALE is opinionated. It assumes Terraform, GCP managed services, and automated CI/CD. Teams wedded to manual processes or different tooling will find friction.

All five pillars takes time. Clients who want speed over structure sometimes push back on the full approach. I've learned to sequence pragmatically — you don't have to solve everything at once — but shortcuts in Security or Automation always surface later. Usually during an audit or outage.

It's a framework, not a product. Every engagement requires real architectural thinking. SCALE provides the lens, not the answer.

Where to Start

The best starting point is a 30-minute architecture review. We look at where you are against the five pillars, identify the most urgent gaps, and map out what a SCALE-driven platform looks like for your specific situation.

If you're mid-growth, mid-audit, or mid-crisis — the sequence matters now.

Work with a GCP specialist — book a free discovery call

Amit Malhotra, Principal GCP Architect, Buoyant Cloud Inc

Work with a GCP specialist — book a free discovery call → https://buoyantcloudtech.com

What a Fractional GCP Architect Actually Does (And When You Need One)

Amit Malhotra — Tue, 03 Mar 2026 02:46:30 +0000

There's a hiring pattern I see a lot in B2B SaaS companies at the 50–200 employee stage: engineering is moving fast, the cloud infrastructure is getting complex, but it's not quite time to hire a full-time Principal Cloud Architect.
That's exactly where fractional cloud architecture fits.
I'm Amit Malhotra, a Principal GCP Architect and founder of Buoyant Cloud. I embed with SaaS engineering teams as a fractional GCP architect — bringing senior-level cloud expertise scoped to what the team actually needs, when they need it.

What fractional actually means in practice
Fractional isn't a watered-down engagement. It's a scoped one.
You get a senior architect who has designed and shipped production GCP infrastructure across industries — not someone learning on your dime. The difference is the model: instead of a full-time hire or a 12-month consulting contract, you get focused, high-impact work tied to your current priorities.
That might look like:

Designing and building a GCP landing zone from scratch
Hardening a GKE cluster ahead of a SOC 2 audit
Replacing service account keys with Workload Identity Federation across your CI/CD pipeline
Investigating and resolving a sudden spike in cloud costs
Setting up a DevSecOps pipeline that your team can actually own and maintain

After the core engagement, many clients keep me on retainer — PR reviews, architecture advisory, and course-correcting before small decisions become expensive problems.

The stack I bring to every engagement
IaC: Terraform with a modular structure — separate modules for networking, GKE, IAM, and services. Terragrunt for DRY configuration across environments.
Identity & Access: Workload Identity Federation over service account keys, always. WIF eliminates an entire class of credential exposure risk at the CI/CD layer.
Networking: Shared VPC with host/service project separation. VPC Service Controls for clients touching regulated data or going through SOC 2. Private Google Access on by default.
Secrets: Secret Manager with versioning and automatic rotation where possible — injected at runtime via WIF, never baked into images or plaintext environment variables.
CI/CD: Cloud Build or GitHub Actions depending on where the team already lives. Artifact Registry for container images. Binary Authorization for production workloads.
Observability: Cloud Monitoring and Cloud Logging as the baseline. Prometheus + Grafana on GKE where teams want more control. OpenTelemetry for instrumentation.

How I structure every engagement — the SCALE Framework
Every engagement I run is grounded in the SCALE Framework:

S — Security by Design (not bolted on post-launch)
C — Cloud-Native architecture (managed services over self-managed where it makes sense)
A — Automation & IaC (everything in code, nothing clicked in console)
L — Lifecycle Ops (Day 2 operations planned from Day 0)
E — Elastic Scalability (design for growth without redesign)

It's the pattern I kept seeing in every successful infrastructure build versus every one that needed to be rebuilt six months later.

What I'm writing about here
Practical, implementation-level content from real engagements:

GKE hardening walkthroughs
Terraform patterns for GCP landing zones
Workload Identity Federation end-to-end
VPC Service Controls deep dives
Kong API Gateway on GKE — real configs
SOC 2 on GCP — mapping controls to actual infrastructure

Long-form guides live on buoyantcloudtech.com/blog.

Let's connect
If you're a CTO or engineering lead evaluating whether fractional cloud architecture is the right fit for your stage — or dealing with a specific GCP challenge right now — I'd love to talk. buoyantcloudtech.com or connect on LinkedIn.