Deploying Prefect Workflows on Cloud Run with Cloud SQL (Production-Ready GCP Setup)

Devops Pilot — Sat, 21 Feb 2026 06:58:39 +0000

If you're running data workflows and don’t want to manage Kubernetes, this architecture might help.

In this project, I built a production-ready workflow system using:

Prefect for orchestration 🔄
Cloud Run for serverless container execution 🚀
Cloud SQL (PostgreSQL) for metadata storage 🗄️
IAM-based authentication between services 🔐

🏗️ Architecture Overview

Prefect Agent → Cloud Run Container → Cloud SQL (Private IP)

Why this approach works well:

Fully serverless compute ⚡
Auto scaling
No VM maintenance
Secure database connectivity
Clean separation of orchestration and execution

🛠️ Key Implementation Steps

Create Cloud SQL instance (Private IP)
Build Docker image for Prefect flow
Push image to Artifact Registry
Deploy to Cloud Run
Configure IAM roles properly
Connect Cloud Run to Cloud SQL

⚠️ Common Mistakes

Forgetting Cloud SQL Auth Proxy
Not configuring VPC connector
Using public IP for database

🎯 Ideal Use Cases

ETL pipelines
Background microservices
Event-driven automation
Internal workflow platforms

🧠 Architecture Check

In a Cloud Run (2nd gen) setup connecting to Cloud SQL via Private IP without a Serverless VPC Connector, what enables network access?

A) Cloud NAT

B) Direct VPC egress configuration

C) Public IP exposure

D) Load Balancer

Answer: B) Direct VPC egress configuration

💡 Interview Tip

If an interviewer asks:

“Why would you choose Cloud Run over GKE for workflows?”

Strong answer:
Cloud Run reduces operational overhead and is ideal for stateless, event-driven workloads. GKE is better when you need fine-grained networking control, sidecars, or complex orchestration.

Full step-by-step guide with commands:

https://devopspilot.com/cloud/gcp/projects/prefect-cloudrun-cloudsql/

DEV Community: Devops Pilot

Deploying Prefect Workflows on Cloud Run with Cloud SQL (Production-Ready GCP Setup)