GCP network egress is the same story as AWS data transfer, just with GCP-specific patterns
I want to share a specific one because it's become more common as more teams run ML workloads
A team running LLM inference on GCP. Vertex AI for model serving, GCS for model storage, Cloud Run for the API layer
Their network costs were $4,100 a month. For their traffic volume it should have been under $1,000
I looked at the egress breakdown
The model weights were being loaded from GCS into the serving infrastructure on every cold start
The model was 14GB. Cloud Run was scaling to zero when there was no traffic. Every scale-up event was downloading 14GB from GCS before the instance could serve a request
At scale, with frequent scale-to-zero events, this was generating enormous egress from GCS. Not because the data was flowing to users. Because the model was being repeatedly loaded from object storage on every infrastructure scale-up
The solution was keeping a minimum number of instances warm so they didn't scale to zero, combined with moving the model weights to a persistent disk attached to the serving infrastructure so the download happened once at startup rather than on every cold start
The change required rethinking the serving architecture slightly. Not a rewrite. A restructuring of how the model was accessed
Monthly network costs after the change: $280
$3,820 saved per month
The insight from this case: ML inference workloads have cost patterns that don't look like traditional web application workloads. The model weights themselves are expensive to move. Architecture decisions need to account for that weight, literally
The standard patterns for web services don't map cleanly to inference. The optimization opportunities are different. And they're often significant
Top comments (0)