Last year, Google flew me to Paris for the announcement of Gemma 3. It was an exciting event. The demos were impressive. But what really mattered h...
For further actions, you may consider blocking this person and/or reporting abuse
Great article. I'm in my deep dive now of finding the fun edges of the model in local, and I feel that pinch of letting something run and getting that surprise later. Thanks and hope you get flown out to somewhere exciting on your next Google visit (hopefully not on the drone ) .
Love it! Thank you for sharing, Daniel!
Great article! Thanks for sharing it.
Could you also share the pricing/cost you saw after running for x amount of time?
Insightful article. I'm literally going to deploy Gemma 4 on Cloud Run and compare it against our Qwen3:8b setup for NER tasks. That's the highest praise I can give : ).
Great job, and thank you for writing this. The community needs more content like this — practical, honest, and actionable.
The VPC point is really interesting, especially the fact that GCS without VPC can actually be slower than Hugging Face for the smaller models. It’s a good reminder that model size isn’t the only variable in inference performance - the network path can become the bottleneck.
We’re looking at similar trade-offs around self-hosted AI inference at SotaTek, especially the balance between GPU utilization, cold starts, latency, and keeping infrastructure costs predictable. The scale-to-zero approach is particularly interesting for workloads that are bursty rather than consistently high-volume.
Curious how you’d approach the same setup for production workloads where a few minutes of cold start isn’t acceptable. Would you keep a minimum instance warm, or move to a different serving architecture once traffic becomes predictable?
Great write-up, Daniel. The finding that the 26B MoE model cold-starts faster than the 2B from HuggingFace is counter-intuitive but makes perfect sense once you think about it — network path and streaming architecture matter more than model size on disk. That VPC Private Google Access detail is the kind of thing that saves people real money and isn't highlighted enough in the docs.
The edge deployment angle you mentioned with drones is particularly interesting. In embodied AI and robotics, you're often running inference on devices with limited connectivity and strict latency constraints. A 4B active-parameter MoE model that reasons and does function calling, running locally via scale-to-zero infra — that's a practical architecture for tactile data processing pipelines where data can't always leave the device. Looking forward to seeing how this stack evolves for those use cases.
Awesome breakdown on optimizing Serverless inference costs for Gemma! 🚀
The pay-per-use architecture on Cloud Run is a game-changer for solo devs and teams looking to deploy custom models without burning through budget on idle GPUs.
From an architecture perspective, combining this with zero-latency client-side preprocessing (e.g. token trimming, prompt sanitization, and structured JSON validation right inside the browser before hitting the Cloud Run endpoint) creates the ultimate cost-effective setup:
Saved this to my bookmarks for my next Gemma project. Thanks for sharing!
I like it, I will try.