DEV Community: Whitney

Apache Geode 2.0: Revival, Reinvention, and the Road Ahead

Whitney — Tue, 03 Mar 2026 18:22:06 +0000

Originally published at https://news.apache.org/foundation/entry/apache-geode-2-0-revival-reinvention-and-the-road-ahead

By: Jinwoo Hwang
Lead Developer, Project Lead, and Release Manager, Apache Geode 2.0
https://JinwooHwang.com

This post is divided into three parts. Part I explains why Apache Geode 2.0 matters. Part II walks through how it was modernized. Part III looks ahead—what we learned, what changed, and how you can help shape what comes next.

Apache Geode 2.0, Part I: The Revival of Apache Geode

Legacy, purpose, and the moment a terminated project came back to life

Apache Geode 2.0 is not just a new release—it is a statement of intent. Before diving into code, frameworks, or version numbers, it is worth understanding why this release exists at all.

This story begins with a platform that once powered mission-critical systems, drifted toward obsolescence, and then found new life through conviction, persistence, and community. This first section sets the stage: the purpose behind the work, the legacy of Apache Geode, and the moment when a seemingly-finished project began its comeback.

I have the privilege of serving as a Committer and Release Manager for Apache Geode 2.0. This release represents one of the most ambitious modernization efforts in the project’s history. For me, it has been more than engineering work—it has been a journey shaped by purpose, responsibility, and a deep belief in the value of our shared open source legacy.

When I stepped into these roles, it became clear that Apache Geode could not survive on incremental change. The Java ecosystem had moved forward—Jakarta EE, Spring, Jetty, Tomcat, and security practices had all evolved—while Geode had effectively stood still. At the same time, unpatched vulnerabilities threatened user trust. To remain relevant, Geode needed a fundamental reset: technically, architecturally, and culturally.

Why I Took on This Project

I do not earn additional compensation for the nights and weekends spent on Apache Geode. I am grateful to my employer for supporting my open source contributions, but this work did not replace my day job. I carried the same responsibilities while taking on this effort.

The reason I stayed with it is simple: purpose. I believe in this project and the community behind it. Friedrich Nietzsche famously wrote, “He who has a why to live can bear almost any how,” an idea later echoed by Viktor Frankl in his work on meaning and resilience. That sense of why—of keeping something valuable alive—carried me through the hardest moments of this journey.

With that context, it is worth stepping back and answering a foundational question.

What Exactly Is Apache Geode?

Apache Geode is a distributed, in‑memory data management platform designed for low‑latency, scalable, and consistent data access. It is built for systems that must react in real time, handle massive data volumes, and remain operational under failure. Data is dynamically partitioned or replicated across a cluster, with built‑in fault tolerance and optional persistence to disk.

As modern applications have shifted toward real-time analytics, event-driven architectures, and microservices, latency has become a central architectural constraint. Disk-backed storage systems, while durable and cost-efficient, often introduce millisecond-scale access times that are incompatible with sub-millisecond response requirements. In-memory data platforms address this gap by keeping active or frequently-accessed data in RAM, significantly reducing access latency and increasing throughput. This approach is particularly important in domains such as financial services, telecommunications, e-commerce, and IoT, where responsiveness, scale, and availability directly influence user experience and operational outcomes.

At its core, Geode aggregates memory, CPU, and network resources across multiple nodes into a single, coherent data fabric. Applications continue running even when individual nodes fail—no blinking, no downtime. Geode supports multiple deployment models, including peer‑to‑peer, client/server, and multi‑site configurations, enabling it to scale from tightly coupled application clusters to geographically distributed systems.

From GemFire to Geode—and Almost to Obsolescence

Apache Geode’s lineage traces back to 2002, when GemStone Systems introduced GemFire, a commercial platform widely used in financial services for real‑time workloads. Through acquisitions—GemStone to SpringSource, then to VMware, and later to Pivotal—the technology evolved before being open sourced in 2015 and donated to The Apache Software Foundation as Apache Geode.

For several years, the project thrived. But after 2019, corporate shifts and changing priorities reduced contributor engagement. By 2022, most committers were inactive. By mid‑2023, development had stopped entirely. In 2024, the PMC voted to terminate the project. Apache Geode appeared to be finished.

Apache Geode Contributors Over Time

Then came 2025. I began upstreaming my internal fork. The community delivered Apache Geode 1.15.2 in September, followed by Apache Geode 2.0 in December. What looked like an ending became a comeback—a transition from long winter to spring.

By the time Apache Geode’s revival began, it was clear that survival alone was not enough. To remain viable, trusted, and relevant, the platform would need far more than incremental fixes—it would need a complete modernization from the ground up.

Learn what Apache Geode is https://geode.apache.org/

How DiDi Scaled to Hundreds of Petabytes with Apache Ozone

Whitney — Thu, 29 Jan 2026 23:42:28 +0000

Building a cost-effective, high-performance data foundation for global mobility

When you’re operating one of the world’s largest ride-hailing and mobility platforms, every millisecond and megabyte counts. For DiDi Global, which generates over one petabyte of new data every day, scaling storage isn’t just a technical challenge—it’s a business imperative.

As the company’s data footprint grew to more than 500PB annually, DiDi’s engineers found themselves battling the limits of their legacy Apache HadoopⓇ Distributed File System (HDFS) storage layer. The infrastructure was struggling to keep pace with the company’s explosive data growth, slowing downstream analytics and machine learning (ML) workloads that power everything from route optimization to dynamic pricing.

The Challenge: Scaling Without Compromise

DiDi’s HDFS-based infrastructure had served the company well, but it was beginning to show its age under the weight of petabyte-scale workloads. The team faced several interconnected problems:

Metadata bottlenecks: File count limits in HDFS created stress on metadata services, driving up latency and throttling performance.
Read-heavy workloads: RPC congestion and HDD I/O bottlenecks introduced lag for analytics and AI pipelines.
Escalating costs: Triple replication inflated storage use and operational expenses.
Operational risk: Even routine maintenance, such as decommissioning, carried stability concerns.

These issues had tangible business impacts. Slow metadata operations increased latency for end users, inflated costs, and created risks during peak demand periods.

“Metadata latency wasn’t just a technical problem—it slowed down business units that rely on real-time analytics and AI insights,” said JiangHua Zhu, Software Engineer, DiDi’s Storage Team.

The Solution: Apache Ozone

After a rigorous evaluation, DiDi selected Apache Ozone™, a next-generation distributed storage system designed for scalability and performance in large, unstructured data environments.

Ozone’s modern architecture—featuring RocksDB-based metadata management, separation of Object Manager (OM) and Storage Container Manager (SCM) services, and containerized data storage—provide the foundation DiDi needed to scale with confidence.

Key Benefits

Massive scalability: Ozone comfortably supports tens of billions of files, removing HDFS metadata constraints.
Performance optimizations: Features like OM Follower Read, multi-cluster routing, and NVMe caching help minimize latency and balance system load.
Cost efficiency through Erasure Coding:
Transitioning from 3x replication to EC 6-3 reduce storage overhead from 3.0x to roughly 1.5x—saving hundreds of petabytes.
Enhanced resilience: Container-based data granularity improves fault tolerance and streamlined operations.

“Ozone gave us the flexibility to scale elastically across hundreds of petabytes without sacrificing performance,” said Wei Ming, DiDi engineer.

The Results: Faster, Leaner, and More Reliable

The move to Apache Ozone delivered measurable, cross-functional benefits across DiDi’s data ecosystem:

Latency: P90 GetMetaLatency improved from 90ms to 17ms.
Throughput: Production read throughput increased by more than 20% with OM follower reads.
Cost savings: Erasure Coding cut the storage footprint nearly in half, saving both capital and operational expenses.
Stability under load: The platform now operates smoothly even during cluster maintenance and peak traffic periods.
Developer productivity: Application teams no longer need to manage small-file compaction, reducing complexity and accelerating data delivery.

Smooth Adoption Through Planning and Community Collaboration

DiDi’s migration to Ozone was meticulous and deliberate. Engineers ensured data consistency with DistCp COMPOSITE_CRC checksums, implemented dual-write for rollback safety, and validated end-to-end compatibility with Hadoop, Apache Spark™, and S3 APIs.

The company also leaned heavily on the Apache Ozone open source community—which contribute bug fixes, performance enhancements, and feedback that benefit all users.

“The open source community was instrumental in our success—we gained support, shared knowledge, and received bug fixes that help everyone,” said Shilun Fan, DiDi’s storage leadership.

DiDi engineers even became active contributors, helping resolve issues such as metadata inconsistencies and Erasure Coding container handling. The collaboration ultimately strengthened both DiDi’s deployment and the broader Ozone ecosystem.

Technical Highlights

Storage savings: Hundreds of petabytes saved through Erasure Coding (6-3).
Read efficiency: 20%+ improvement from OM follower reads and NVMe caching.
Unified access: Hadoop API and S3 compatibility for batch, interactive, and ML workloads.
Scalability: A single Ozone cluster can handle ~5 billion files, with the potential to scale to tens of billions.

Looking Ahead

DiDi’s storage team continues to push the boundaries of performance and efficiency. Upcoming initiatives include:

Integrating IO_URING and SPDK to enhance I/O performance.
Developing AI-driven operational insights for anomaly detection and auto-remediation.
Piloting tiered storage strategies for hot, warm, and cold data layers to optimize cost and performance.

“Ozone is more than a storage layer—it’s the backbone of DiDi’s data ecosystem and future AI innovation,” said Hongbing Wang, DiDi technical lead.

The Takeaway

By embracing Apache Ozone, DiDi transformed its data storage infrastructure from a limitation into a competitive advantage. The move delivered lower costs, higher reliability, and faster access to the insights that power intelligent mobility.

At petabyte scale, even incremental improvements deliver outsized impact—and with Apache Ozone, DiDi has built a storage foundation ready for the next decade of data-driven innovation.

To learn more about Apache Ozone:

Apache Ozone GitHub: https://github.com/apache/ozone
Apache Ozone Getting Started: https://ozone.apache.org/docs/edge/start/startfromdockerhub.html
Apache Ozone LinkedIn page: https://www.linkedin.com/company/apache-ozone/
Apache Ozone X.com handle: https://x.com/ApacheOzone
Apache Ozone Best Practices at Didi: https://ozone.apache.org/assets/ApacheOzoneBestPracticesAtDidi.pdf