DEV Community

Cover image for How to Actually Lock Down Your Cloud Run Services (Not Just Pretend To)
Chinmay Jain
Chinmay Jain

Posted on • Originally published at gamechanger1s.hashnode.dev

How to Actually Lock Down Your Cloud Run Services (Not Just Pretend To)

#google #networking #serverless #security

(Namaste)! 🙏

The Problem We're Solving

You've built something awesome on Google Cloud Run. Maybe it's a smoothie recipe API, a coffee service, or any cool app. It works great! You added login, API keys, and all that security stuff.

But here's the scary truth: Your API is still sitting on the public internet like a house with the address posted everywhere.

Sure, you locked the doors (added auth). But everyone can still see your house, walk up to it, and try to kick the door down. Hackers can spam it with thousands of fake requests. Bots can probe it for weak spots.

Google even admits this is a problem in their own bug tracker (https://issuetracker.google.com/issues/237250997).

What if your APIs could be completely invisible? Like they don't even exist on the public internet?

That's what we're building today. No more exposed services. No more public endpoints. Just a single, secure gateway that acts as the ONLY door to your private services.

What We're Building

Think of it like a high-security nightclub with VIP rooms:

🏢 Your Cloud Run services = VIP rooms upstairs (nobody even knows they exist)

🚪 The Load Balancer = The club's main entrance with a bouncer

🛡️ IAP (Identity-Aware Proxy) = The bouncer checking IDs at the door

🎫 Your API Gateway = The ONE person with a valid VIP pass

Here's The High-Level Architecture:

The Flow Explained:

Step 1: User sends a request to your API Gateway with their Firebase token (proof they're legit)

Step 2: API Gateway creates an IAP JWT - think of this as a special VIP wristband that says "this request is from our trusted gateway"

Step 3: API Gateway walks up to the Load Balancer entrance, wristband in hand

Step 4: IAP (the bouncer) checks the wristband → "Yep, this is legit. You're on the list."

Step 5: Load Balancer routes the request to your private Cloud Run service (smoothie-service, coffee-service, or whatever you built)

Step 6: Response comes back through the same secure path

See those private services at the bottom of the diagram? They're completely hidden from the world. The Load Balancer is the ONLY thing that can talk to them.

Everyone else? They can't even see the entrance exists.

Let's build this fortress.

Step 1: Hide Your Services From the Internet

We're going to make your Cloud Run services private. Like, actually invisible to the world.

For this guide, we'll use the services from our diagram:

  • smoothie-service - Returns smoothie recipes and nutrition info
  • coffee-service - Shares coffee brewing tips and cafe recommendations

Making Them Invisible

  1. Open Cloud Run in your Google Cloud console
  2. Click on smoothie-service
  3. Hit "Edit & Deploy New Revision"
  4. Go to the "Networking" tab
  5. Find "Ingress" and change it:
    • ❌ OLD: Allow all traffic (visible to everyone on the internet)
    • ✅ NEW: Allow internal traffic and traffic from Cloud Load Balancing (hidden, internal only)
  6. Click "Deploy"

Did It Actually Work?

Try visiting your service directly:

curl https://smoothie-service-xxxxx.run.app
Enter fullscreen mode Exit fullscreen mode

You should get: 403 Forbidden 🎉

That's GOOD! It means your service just vanished from the public internet. Nobody can reach it anymore.

Now do the exact same thing for coffee-service.

Quick heads-up: If you have a service that handles CORS preflight stuff (those OPTIONS requests from browsers), keep that one public. Browsers need direct access to it.

Step 2: Build Your Secret Entrance (The Load Balancer)

Now we create the ONE secure door that leads to your hidden services.

2.1. Create Service Connectors (NEGs)

NEGs are like secret tunnels. They tell the Load Balancer "hey, here's how to find this hidden service."

For Smoothie Service:

  1. Go to Compute Engine → Network endpoint groups
  2. Click "Create network endpoint group"
  3. Fill it out:
    • Name: smoothie-service-neg
    • Type: Serverless network endpoint group
    • Region: us-central1 (or wherever your service lives)
    • Serverless type: Cloud Run
    • Service: Pick smoothie-service
  4. Hit "Create"

For Coffee Service: Do the exact same thing but call it coffee-service-neg.

These NEGs are now the secret tunnels connecting your Load Balancer to your hidden services. Nobody else knows they exist.

2.2. Build the Main Entrance (Load Balancer)

This is the fun part. We're building the fortress entrance.

  1. Go to Network Services → Load balancing
  2. Click "Create load balancer"
  3. Pick "Application Load Balancer (HTTP/HTTPS)"

The Front Door Setup

  • Protocol: HTTPS (always use HTTPS!)
  • IP Address: Click "Reserve new static IP" and name it secure-api-entrance
    • 📝 Write this IP address down! You'll need it soon.
  • Certificate: Click "Create new certificate" and pick "Google-managed"
  • Domain: Type your domain like api.myawesomeapp.com

Connecting to Your Hidden Services

Now we tell the Load Balancer where your secret services are hiding.

For Smoothie Service:

  1. Click "Create a backend service"
  2. Name: smoothie-service-backend
  3. Backend type: Serverless NEG
  4. Backend: Select smoothie-service-neg (that tunnel we created)
  5. Click "Create"

For Coffee Service: Same thing! Create coffee-service-backend using coffee-service-neg.

Setting Up the Routes

This tells the Load Balancer "when someone asks for smoothies, send them to the smoothie service."

If URL path is... Send to...
/smoothie smoothie-service-backend
/coffee coffee-service-backend

Click "Create" and grab a coffee. ☕ This takes a few minutes to set up.

Step 3: Point Your Domain at the New Entrance

Remember that IP address we wrote down? Time to use it!

  1. Go to wherever you manage your domain (GoDaddy, Cloudflare, Route 53, etc.)
  2. Create a new A record:
    • Name: api (this creates api.myawesomeapp.com)
    • Points to: That IP address from Step 2 (secure-api-entrance)

⏰ Wait about 10-20 minutes. DNS changes are like waiting for the DMV - they take time. The SSL certificate also needs to activate.

While you wait, maybe think about your favorite coffee order? ☕

Step 4: Add the Bouncer (IAP Security)

This is where the magic happens. We're adding a security bouncer that only lets your API Gateway through.

4.1. First-Time Setup (OAuth Screen)

Google needs you to set this up once. It's quick:

  1. Go to APIs & Services → OAuth consent screen
  2. User Type: Pick "Internal"
  3. App name: Type something like "My Secure APIs"
  4. Support email: Your email
  5. Keep clicking "Save and Continue" until you're done

This is just Google covering their legal bases. Don't overthink it.

4.2. Turn On the Bouncer

  1. Go to Security → Identity-Aware Proxy
  2. You'll see a list of your backend services
  3. Flip the switch to ON for:
    • smoothie-service-backend
    • coffee-service-backend

Boom. The bouncer is now standing guard at your entrance.

4.3. Put Your API Gateway on the VIP List

Now we tell the bouncer "hey, this one person is allowed in - let them through."

  1. Still on the IAP page, click smoothie-service-backend
  2. A side panel opens → Click "Add Principal"
  3. New principals: Paste your API Gateway's service account email (Looks like: my-gateway-sa@myproject.iam.gserviceaccount.com)
  4. Role: Choose IAP-secured Web App User
  5. Click "Save"

Do this for coffee-service-backend too.

Your gateway is now on the VIP list. Everyone else gets turned away at the door.

4.4. Get the Secret Password (Client ID)

This is the special password your API Gateway will show the bouncer to prove it's legit.

  1. Go to APIs & Services → Credentials
  2. Look for something called "IAP-App" (Google created it automatically when you turned on IAP)
  3. Copy the Client ID - it's a long string like: 
   987654321-abcdefghijklmnop.apps.googleusercontent.com
Enter fullscreen mode Exit fullscreen mode

💾 SAVE THIS SOMEWHERE SAFE! You'll need it in the next step.

Step 5: Give Your API Gateway the Secret Password

Now we update your API Gateway config so it knows the secret password to get past the bouncer.

What Your Config Looks Like Now (Not Secure) 

/smoothie:
  get:
    x-google-backend:
      address: "https://smoothie-service-xxxxx.run.app"
      deadline: 30
Enter fullscreen mode Exit fullscreen mode

This goes straight to the service. Anyone with the URL can hit it. Not good.

What We're Changing It To (Actually Secure!) 

/smoothie:
  get:
    x-google-backend:
      address: "https://api.myawesomeapp.com/smoothie"
      deadline: 30
      jwt_audience: "987654321-abcdefghijklmnop.apps.googleusercontent.com"
Enter fullscreen mode Exit fullscreen mode

See what changed?

✅ Now points to the Load Balancer (not directly to the service)

✅ Includes the secret password in jwt_audience

When your gateway makes a request, it automatically creates a special token (JWT) with this audience. The bouncer checks this token and says "yep, you're good!"

Your Complete Config File

Here's what your openapi.yaml should look like:

swagger: "2.0"
info:
  title: My Awesome Secure API
  description: Locked down with IAP - Fort Knox level security
  version: 1.0.0
host: my-gateway-xxxxx.gateway.dev
schemes:
  - https
paths:
  /smoothie:
    get:
      summary: Get smoothie recipes and nutrition info
      operationId: getSmoothie
      responses:
        '200':
          description: Smoothie data delivered
      x-google-backend:
        address: "https://api.myawesomeapp.com/smoothie"
        deadline: 30
        jwt_audience: "987654321-abcdefghijklmnop.apps.googleusercontent.com"
  /coffee:
    get:
      summary: Get coffee brewing tips and recommendations
      operationId: getCoffee
      responses:
        '200':
          description: Coffee tips incoming
      x-google-backend:
        address: "https://api.myawesomeapp.com/coffee"
        deadline: 30
        jwt_audience: "987654321-abcdefghijklmnop.apps.googleusercontent.com"
Enter fullscreen mode Exit fullscreen mode

Save this and deploy it to your API Gateway.

Step 6: The Moment of Truth (Testing Time!)

Alright, let's see if we actually built a fortress or just a paper wall. Run these three tests:

❌ Test 1: Try Sneaking in the Back Door 

curl https://smoothie-service-xxxxx.run.app
Enter fullscreen mode Exit fullscreen mode

You should get: 403 Forbidden

What this means: "Nope! This service is invisible. Nice try, hacker."

Your service is completely hidden from the public internet. ✅

❌ Test 2: Try the Front Door Without a Pass 

curl https://api.myawesomeapp.com/smoothie
Enter fullscreen mode Exit fullscreen mode

You should get: An error like "Invalid IAP credentials: empty token"

What this means: "The bouncer says no. You're not on the VIP list."

The Load Balancer is locked down. Random people can't just walk in. ✅

✅ Test 3: Use the Proper VIP Entrance 

curl https://my-gateway-xxxxx.gateway.dev/smoothie
Enter fullscreen mode Exit fullscreen mode

You should get: 200 OK with actual smoothie data! 🥤

What this means: "Welcome! Your gateway showed the right pass. Come on in!"

Everything works perfectly through the secure path. ✅

What You Just Built (Seriously, Pat Yourself on the Back)

🎉 Dude, you just built enterprise-level security! Here's what you accomplished:

✅ Your services are completely hidden from the internet

✅ Nobody can even ping them directly

✅ The ONLY way in is through your API Gateway

✅ Even if someone finds the Load Balancer URL, they can't get through without the secret password

✅ You have one single entrance to monitor and control

Quick Recap (For Future You)

When you come back to this in 6 months and think "wait, how did I set this up?":

  1. Made services private → Changed Cloud Run ingress to internal-only
  2. Created NEGs → Secret tunnels connecting Load Balancer to services
  3. Built Load Balancer → The single entrance to everything
  4. Turned on IAP → Added the security bouncer
  5. Added Gateway to VIP list → Gave it permission to pass
  6. Updated Gateway config → Added the secret password (jwt_audience)

That's it. Six steps to lock everything down tight.

Wait... there's one more thing.

What if you want your private Cloud Run services to communicate with each other internally using their direct private service endpoints, without going through the API Gateway at all?

What if developers want to call the Cloud Run endpoints directly via a local host machine without going through the API Gateway at all?

What if you want to access your Cloud Run services through a private VPN tunnel where literally nobody else can even see the entrance?

Yeah, that's possible. And it's actually pretty cool.

🚀 Coming Next: How-To: Access Internal Cloud Run Services via VPN (e.g., Tailscale)

Spoiler: Your services will be so hidden, they'll basically be in a secret underground bunker. Stay tuned. 🔒🕵️

Originally published at gamechanger1s.hashnode.dev_

Top comments (0)