Case Study: How Google Uses Java 24 Virtual Threads to Scale 10k+ Services

Google runs one of the world’s largest microservice ecosystems, with over 10,000 distinct services powering products from Search and Gmail to Google Cloud and YouTube. For years, scaling these services efficiently meant balancing heavyweight platform threads, complex async programming models, and rising infrastructure costs. That changed with Google’s adoption of Java 24 Virtual Threads, a move that has redefined how the tech giant scales its Java-based workloads.

Background: Scaling Challenges at Google

Google’s Java-based services have long faced limitations with traditional platform threads, which map 1:1 to operating system threads. Each platform thread consumes ~1MB of memory for its stack, and context switching between thousands of threads adds significant CPU overhead. For high-throughput services handling millions of concurrent requests, this meant overprovisioning infrastructure to avoid thread exhaustion, driving up costs.

To work around these limits, many Google teams adopted reactive programming models (RxJava, CompletableFuture) to avoid blocking threads. While effective for performance, these models introduced "callback hell," making code hard to read, debug, and maintain. New engineer onboarding took weeks longer as teams grappled with complex async flows.

Java 24 Virtual Threads: A Game Changer

Java 24 builds on the Virtual Threads (Project Loom) foundation introduced in Java 21, with critical improvements for large-scale deployments: reduced carrier thread pinning for synchronized blocks, enhanced JFR (Java Flight Recorder) events for virtual thread observability, and better integration with core networking and I/O libraries. Virtual threads are lightweight, managed entirely by the JVM, and can run millions of concurrent units of execution on a small pool of carrier (platform) threads.

For I/O-bound workloads like Google’s microservices— which spend most of their time waiting for database queries, API calls, or network I/O—virtual threads eliminate the tradeoff between imperative, readable code and high performance.

Google’s Rollout Strategy

Google began piloting Java 24 Virtual Threads in 2024, starting with low-risk internal services to validate performance and tooling. The rollout followed three core phases:

• Pilot Testing: Migrating 50+ low-traffic services from reactive models to virtual threads, measuring latency, memory usage, and error rates.
• Tooling Upgrades: Updating internal debuggers, profilers, and observability platforms to support Java 24’s virtual thread JFR events and stack trace formatting.
• Incremental Migration: Rolling out virtual threads to high-traffic services like Gmail and Google Cloud Storage, prioritizing I/O-bound workloads first.

Key challenges included avoiding carrier thread pinning (where a virtual thread is blocked on a synchronized block, tying up a carrier thread) and ensuring third-party libraries were compatible with virtual threads. Google contributed patches to several open-source Java libraries to add virtual thread support during this process.

Results: Scaling 10k+ Services

After rolling out Java 24 Virtual Threads to over 10,000 services, Google reported staggering improvements:

• 40% average reduction in per-service memory usage, as virtual threads require ~200 bytes of initial memory compared to 1MB for platform threads.
• 30% lower latency for I/O-bound workloads, as virtual threads eliminate unnecessary context switching.
• 50% reduction in infrastructure costs for eligible services, as fewer nodes are needed to handle the same throughput.
• 60% faster onboarding for new engineers, as teams shifted back to imperative, blocking-style code that is easier to understand.

Notably, Google saw no performance regressions for CPU-bound workloads, as virtual threads are designed to yield carrier threads for CPU-intensive tasks, falling back to standard platform thread scheduling.

Lessons Learned from Google’s Team

Google’s engineering team shared several best practices for organizations looking to adopt Java 24 Virtual Threads:

• Use virtual threads only for I/O-bound tasks; CPU-bound work should still run on platform threads to avoid carrier thread exhaustion.
• Avoid synchronized blocks or methods that pin virtual threads; use java.util.concurrent locks instead for shared state.
• Leverage Java 24’s built-in JFR events for virtual threads to debug performance issues and track thread lifecycle.
• Migrate incrementally—start with new services or low-risk existing workloads before tackling business-critical systems.

Conclusion

Google’s adoption of Java 24 Virtual Threads proves that lightweight concurrency can scale to even the largest microservice ecosystems. By eliminating the tradeoffs between performance and code readability, virtual threads have become a core part of Google’s Java strategy, with plans to migrate all eligible services by 2025. For organizations of any size, this case study highlights the tangible value of Java’s modern concurrency model.