Never Get a Surprise GCP Bill Again: Budget Alerts as Code with Terraform 🚨

A student got a $55K GCP bill from a leaked API key. A dev left a GPU instance running over the weekend — $3,200 gone. Setting a budget in the console takes 5 minutes but nobody does it. Here's how to make it impossible to forget — deploy budget alerts, Slack notifications, and an automatic billing kill switch with Terraform.

GCP does not cap your spending by default. There's no hard limit. Your billing account is an open credit line to Google, and if something goes wrong — a runaway Cloud Function, a misconfigured autoscaler, a leaked service account key — you won't know until the invoice arrives. 😱

The fix? Budgets, alerts, and automated responses — all deployed as code so every project gets them from day one.

💸 The 3 Layers of Cost Protection

Most teams stop at Layer 1. That's why they still get surprised.

Layer	What It Does	Response Time
Budget Alerts (Email)	Emails billing admins at thresholds	Hours (someone reads the email)
Pub/Sub → Slack	Posts to your team channel instantly	Minutes (someone sees Slack)
Auto-Disable Billing	Cloud Function kills the billing account	Seconds (fully automated) 🤖

Let's deploy all three.

🔧 Layer 1: Budget Alerts with Terraform

The google_billing_budget resource is the foundation. This sets up email alerts at 50%, 80%, and 100% of your monthly budget:

resource "google_billing_budget" "monthly" {
  billing_account = var.billing_account_id
  display_name    = "${var.project_id}-monthly-budget"

  budget_filter {
    projects               = ["projects/${var.project_number}"]
    credit_types_treatment = "INCLUDE_ALL_CREDITS"
  }

  amount {
    specified_amount {
      currency_code = "USD"
      units         = var.monthly_budget  # e.g., 1000 for $1,000
    }
  }

  # Alert at 50%, 80%, 100% of actual spend
  threshold_rules {
    threshold_percent = 0.5
    spend_basis       = "CURRENT_SPEND"
  }
  threshold_rules {
    threshold_percent = 0.8
    spend_basis       = "CURRENT_SPEND"
  }
  threshold_rules {
    threshold_percent = 1.0
    spend_basis       = "CURRENT_SPEND"
  }

  # Alert at 90% of FORECASTED spend (early warning)
  threshold_rules {
    threshold_percent = 0.9
    spend_basis       = "FORECASTED_SPEND"  # 👈 This is the magic one
  }
}

variable "monthly_budget" {
  type        = number
  description = "Monthly budget in USD"
}

variable "billing_account_id" {
  type = string
}

variable "project_number" {
  type = string
}

⚠️ Critical gotcha: FORECASTED_SPEND alerts you before you hit the limit by projecting current trends to end of month. Most teams only set CURRENT_SPEND alerts and get notified after the money is already gone. Always include at least one forecasted threshold.

Savings: $0 — this is free. Budget alerts cost nothing. There's zero excuse not to have them. 🎯

🔔 Layer 2: Pub/Sub + Slack Notifications

Email alerts get buried. Slack alerts get seen. Here's the full pipeline:

Budget → Pub/Sub Topic → Cloud Function → Slack Webhook

Step 1: Create the Pub/Sub topic and wire it to the budget

resource "google_pubsub_topic" "budget_alerts" {
  name    = "budget-alerts"
  project = var.project_id
}

# Update the budget to publish to Pub/Sub
resource "google_billing_budget" "monthly_with_pubsub" {
  billing_account = var.billing_account_id
  display_name    = "${var.project_id}-monthly-budget"

  budget_filter {
    projects               = ["projects/${var.project_number}"]
    credit_types_treatment = "INCLUDE_ALL_CREDITS"
  }

  amount {
    specified_amount {
      currency_code = "USD"
      units         = var.monthly_budget
    }
  }

  threshold_rules {
    threshold_percent = 0.5
    spend_basis       = "CURRENT_SPEND"
  }
  threshold_rules {
    threshold_percent = 0.8
    spend_basis       = "CURRENT_SPEND"
  }
  threshold_rules {
    threshold_percent = 1.0
    spend_basis       = "CURRENT_SPEND"
  }
  threshold_rules {
    threshold_percent = 0.9
    spend_basis       = "FORECASTED_SPEND"
  }

  all_updates_rule {
    pubsub_topic                     = google_pubsub_topic.budget_alerts.id
    disable_default_iam_recipients   = false  # Keep email alerts too
  }
}

Step 2: Deploy a Cloud Function that posts to Slack

# Cloud Function source
resource "google_storage_bucket" "function_source" {
  name     = "${var.project_id}-budget-fn-source"
  location = "US"
}

resource "google_storage_bucket_object" "function_zip" {
  name   = "budget-alert-slack.zip"
  bucket = google_storage_bucket.function_source.name
  source = "${path.module}/functions/budget-alert-slack.zip"
}

resource "google_cloudfunctions2_function" "budget_slack" {
  name     = "budget-alert-to-slack"
  location = var.region

  build_config {
    runtime     = "python312"
    entry_point = "handle_budget_alert"
    source {
      storage_source {
        bucket = google_storage_bucket.function_source.name
        object = google_storage_bucket_object.function_zip.name
      }
    }
  }

  service_config {
    max_instance_count = 1
    available_memory   = "256M"
    timeout_seconds    = 60

    environment_variables = {
      SLACK_WEBHOOK_URL = var.slack_webhook_url
    }
  }

  event_trigger {
    trigger_region = var.region
    event_type     = "google.cloud.pubsub.topic.v1.messagePublished"
    pubsub_topic   = google_pubsub_topic.budget_alerts.id
  }
}

The Cloud Function code (Python):

# main.py
import base64
import json
import os
import urllib.request

def handle_budget_alert(cloud_event):
    """Triggered by Pub/Sub budget notification."""
    data = base64.b64decode(cloud_event.data["message"]["data"]).decode()
    notification = json.loads(data)

    cost    = notification.get("costAmount", 0)
    budget  = notification.get("budgetAmount", 0)
    percent = (cost / budget * 100) if budget > 0 else 0

    # Pick emoji based on severity
    if percent >= 100:
        emoji = "🔴"
    elif percent >= 80:
        emoji = "🟠"
    else:
        emoji = "🟡"

    message = {
        "text": (
            f"{emoji} *GCP Budget Alert*\n"
            f"• Spent: *${cost:,.2f}* of ${budget:,.2f} "
            f"(*{percent:.0f}%*)\n"
            f"• Project: {notification.get('budgetDisplayName', 'Unknown')}"
        )
    }

    webhook_url = os.environ["SLACK_WEBHOOK_URL"]
    req = urllib.request.Request(
        webhook_url,
        data=json.dumps(message).encode(),
        headers={"Content-Type": "application/json"},
    )
    urllib.request.urlopen(req)

Now your team sees this in Slack the moment a threshold is crossed:

🟠 GCP Budget Alert
• Spent: $812.50 of $1,000.00 (81%)
• Project: payment-api-prod-monthly-budget

☠️ Layer 3: The Kill Switch (Non-Prod Only)

For dev/staging/sandbox projects, you can go nuclear — automatically disable billing when the budget is exceeded. This is Google's own recommended pattern.

# kill_switch.py — ONLY for non-production projects
import base64
import json
import os
from googleapiclient import discovery

def kill_billing(cloud_event):
    data = base64.b64decode(cloud_event.data["message"]["data"]).decode()
    notification = json.loads(data)

    cost   = notification.get("costAmount", 0)
    budget = notification.get("budgetAmount", 0)

    if cost <= budget:
        return  # Under budget, do nothing

    project_id = os.environ["GOOGLE_CLOUD_PROJECT"]
    billing = discovery.build("cloudbilling", "v1", cache_discovery=False)

    # Disable billing — THIS KILLS ALL SERVICES IN THE PROJECT
    billing.projects().updateBillingInfo(
        name=f"projects/{project_id}",
        body={"billingAccountName": ""},  # 👈 Empty = unlink billing
    ).execute()

    print(f"💀 Billing disabled for {project_id} — cost ${cost} exceeded ${budget}")

⚠️ WARNING: Disabling billing terminates ALL services in the project, including free-tier ones. Google doesn't guarantee data preservation. NEVER use this on production. Only for dev/staging/sandbox where you'd rather lose the environment than get a surprise bill.

Terraform for the kill switch:

resource "google_cloudfunctions2_function" "kill_switch" {
  count    = var.environment == "prod" ? 0 : 1  # 👈 Never in prod!
  name     = "budget-kill-switch"
  location = var.region

  build_config {
    runtime     = "python312"
    entry_point = "kill_billing"
    source {
      storage_source {
        bucket = google_storage_bucket.function_source.name
        object = google_storage_bucket_object.kill_switch_zip.name
      }
    }
  }

  service_config {
    max_instance_count = 1
    available_memory   = "256M"
    timeout_seconds    = 60
  }

  event_trigger {
    trigger_region = var.region
    event_type     = "google.cloud.pubsub.topic.v1.messagePublished"
    pubsub_topic   = google_pubsub_topic.budget_alerts.id
  }
}

The count = var.environment == "prod" ? 0 : 1 is your safety net — this function literally cannot exist in a production project. 🛡️

📊 The Multi-Project Budget Matrix

Real companies don't have one project. They have dozens. Here's how to budget all of them from a single Terraform module:

variable "project_budgets" {
  type = map(object({
    project_number = string
    monthly_budget = number
    environment    = string
  }))
  default = {
    "payment-api-prod" = {
      project_number = "123456789"
      monthly_budget = 5000
      environment    = "prod"
    }
    "payment-api-dev" = {
      project_number = "987654321"
      monthly_budget = 500
      environment    = "dev"
    }
    "ml-pipeline-staging" = {
      project_number = "456789123"
      monthly_budget = 1000
      environment    = "staging"
    }
  }
}

resource "google_billing_budget" "per_project" {
  for_each        = var.project_budgets
  billing_account = var.billing_account_id
  display_name    = "${each.key}-budget"

  budget_filter {
    projects               = ["projects/${each.value.project_number}"]
    credit_types_treatment = "INCLUDE_ALL_CREDITS"
  }

  amount {
    specified_amount {
      currency_code = "USD"
      units         = each.value.monthly_budget
    }
  }

  threshold_rules { threshold_percent = 0.5; spend_basis = "CURRENT_SPEND" }
  threshold_rules { threshold_percent = 0.8; spend_basis = "CURRENT_SPEND" }
  threshold_rules { threshold_percent = 1.0; spend_basis = "CURRENT_SPEND" }
  threshold_rules { threshold_percent = 0.9; spend_basis = "FORECASTED_SPEND" }

  all_updates_rule {
    pubsub_topic = google_pubsub_topic.budget_alerts.id
  }
}

Add a new project? Add one entry to the map. terraform apply. Done. Every project gets identical protection. ✅

💡 Quick Reference: What to Deploy First

Layer	Effort	Cost	Impact
Budget alerts (email)	5 min	Free	Baseline visibility
Forecasted spend alert	2 min	Free	Early warning before overspend
Pub/Sub → Slack	20 min	~$0/month (free tier)	Team-wide awareness
Kill switch (non-prod)	15 min	~$0/month (free tier)	Automatic cost cap
Multi-project budget map	10 min	Free	Org-wide protection

Start with budget alerts. They're free, they take 5 minutes, and they're the single best thing you can do to avoid a surprise bill. 🎯

📊 TL;DR

Budget alerts      = FREE, takes 5 min, no excuse to skip
FORECASTED_SPEND   = warns you BEFORE you hit the limit (most teams miss this)
Pub/Sub → Slack    = real-time team visibility, pennies/month
Kill switch        = auto-disable billing for dev/staging (NEVER prod)
for_each budgets   = one Terraform map protects every project
Billing delay      = 24hrs+ lag, so set budgets BELOW your true limit
No hard cap exists = GCP will never stop charging you automatically

Bottom line: GCP will happily charge you $50K while you sleep. Budget alerts are free, take 5 minutes, and are the only thing standing between you and a career-ending invoice. Deploy them now. 🔥

---

Your dev project doesn't have a budget alert yet, does it? Go deploy that google_billing_budget resource right now — it's free and takes 5 minutes. Your future self (and your CFO) will thank you. 😀

Found this helpful? Follow for more GCP cost optimization with Terraform! 💬