Segment Replication vs Document Replication: The OpenSearch 2.x Architecture Choice

When you write a document into OpenSearch, the primary shard accepts it, indexes it, and then makes sure every replica shard ends up with the same data. For years there was exactly one way to do that: replay the operation on every replica. OpenSearch 2.x changed the game by introducing segment replication, a fundamentally different strategy where replicas copy the already-built index segments instead of re-executing every write.

This choice sounds like an internal implementation detail, but it reshapes how your cluster behaves under load. It changes indexing throughput, CPU consumption, network traffic patterns, and how you think about consistency. In this post I will break down both models, show what actually happens on each node, and give you a practical framework for choosing between them.

The Baseline: How Replication Works in OpenSearch

Every index in OpenSearch is split into shards. Each shard has one primary copy and zero or more replica copies. The primary is the write authority: every indexing, update, and delete operation goes to the primary first. Replicas exist for two reasons: high availability (if a node dies, a replica gets promoted) and read scaling (search queries can be served by replicas, spreading the load).

The question this post is about is narrower but critical: after the primary processes a write, how does that write reach the replicas?

Document Replication: Replaying Every Operation

Document replication is the classic model, inherited from Elasticsearch and the default for many years. Here is the flow:

1. The coordinating node forwards the bulk request to the primary shard.
2. The primary indexes each document into its in-memory buffer and appends the operation to its translog (the write-ahead log).
3. Once the primary has persisted the operation to its translog, it forwards the same operation to every replica shard in parallel.
4. Each replica performs the exact same indexing operation: parse the document, run it through the analysis chain, add terms to the inverted index, update doc values, write to its own translog.
5. When enough replicas have acknowledged (based on your wait_for setting), the primary confirms the write to the client.

The key insight: every replica independently rebuilds the index structure from the raw operations. If you index one million documents, every replica tokenizes, analyzes, and inverts one million documents.

Why this model is expensive

Indexing is CPU-heavy. Analysis chains, term dictionaries, and segment construction burn cycles. With document replication, you pay that CPU cost on every replica. On a cluster with one primary and two replicas, you do the indexing work three times.

It also couples read and write amplification. If you add replicas to serve more search traffic, you silently increase the indexing cost of every write. Clusters that are write-heavy (log ingestion, metrics pipelines) end up spending a surprising share of their CPU on redundant re-indexing.

There is a second, subtler cost: inconsistencies between replicas. Because each replica indexes independently, small differences can creep in: segment layouts differ, merge timing differs, and occasionally a replica can fail an operation the primary succeeded on. OpenSearch has recovery mechanisms, but the model inherently produces slightly divergent shard states that must be reconciled.

Segment Replication: Copying the Finished Product

Segment replication flips the model. Instead of sending the raw document operations to replicas, the primary sends the finished Lucene segments.

Recall how indexing works on the primary: documents accumulate in an in-memory buffer, and every second (by default) a refresh creates a new immutable Lucene segment. Segments are the actual files on disk that hold the inverted index, stored fields, doc values, and term vectors.

With segment replication:

1. The primary indexes documents as usual: buffer, refresh, build segments.
2. After a flush (when segments are written to disk and the translog is trimmed), the primary notifies replicas that new segments are available.
3. Replicas copy the segment files directly from the primary over the network, using the same file-level replication machinery OpenSearch already uses for shard recovery.
4. Replicas load the copied segments and serve searches from them. They never run the analysis or inversion logic themselves.

Why this is a big deal

The CPU cost of indexing is now paid once, on the primary. Replicas become pure consumers of segment files: their job is disk I/O and network transfer, not analysis and inversion. On write-heavy workloads, the savings are dramatic. OpenSearch benchmarks have shown multi-x improvements in indexing throughput and corresponding drops in CPU usage per document when switching to segment replication.

The consistency story also improves. Because replicas hold byte-identical segment files built by the primary, there is exactly one index structure in the cluster per shard, not N slightly-different ones. Relevancy scoring, aggregation results, and sort orders become deterministic across copies.

The Trade-offs Nobody Tells You About

Segment replication is not free lunch. Understanding the costs is where the real engineering decision lives.

1. Replication lag is coarser-grained

Document replication replicates operations continuously, so replicas are typically within milliseconds of the primary (bounded by the network round trip). Segment replication only ships data when segments are flushed to disk. A flush happens on a schedule or when the translog fills up, so replicas can lag by the flush interval.

For most search workloads a lag of a few seconds is invisible. But if your application reads its own writes, for example a user updates a profile and immediately reloads the page, a replica serving that read might not have the new segment yet. You will need routing discipline: send read-after-write traffic to the primary, or accept and document the lag.

2. Network bandwidth moves from small and frequent to large and bursty

Document replication sends compact operation streams: document JSON plus metadata. Segment replication sends whole segment files, which include all the index structures. The total bytes over time are usually comparable or better (segments are compressed and deduplicated at the file level), but the traffic pattern changes from a steady trickle of small requests to periodic bursts of large file transfers.

On clusters with many shards per node, flush events can cluster together and saturate network links. This matters on cloud instances with burst-credit networking or on clusters sharing network bandwidth with other tenants.

3. Recovery semantics change

With document replication, a recovering replica can catch up by replaying recent operations from the translog. With segment replication, recovery means copying segment files, which is generally fast because OpenSearch reuses unchanged files, but the model changes how you reason about node restarts and shard relocation. In practice file-level reuse makes this efficient, but it is a different operational profile worth knowing before you flip the switch.

4. It is the strategic direction of the project

OpenSearch has been explicit that segment replication is the future. Major features, including future work on remote-backed storage and decoupled compute, assume the segment-copy model. Choosing document replication today is choosing the legacy path. That does not make it wrong for every workload, but it is a factor in long-term planning.

Choosing Between Them: A Practical Framework

Here is how I think about the decision in practice:

Choose segment replication when:

• Your workload is write-heavy or indexing-CPU-bound: log ingestion, observability pipelines, metrics, event streams.
• You run large clusters where replica count is high and redundant indexing cost dominates.
• You want deterministic, identical results across primary and replicas.
• You are building for the long term and want alignment with the project's roadmap.

Stay on document replication when:

• You need the tightest possible replication lag and read-your-writes guarantees (and cannot or do not want to route reads to primaries).
• Your workload is read-dominated with modest indexing rates, so the CPU savings do not matter.
• You depend on plugins or features that still assume document replication semantics (check the docs for your version; coverage has expanded rapidly in the 2.x line).

Either way, measure. Run a realistic indexing benchmark with your own documents, analysis chains, and replica configuration. The crossover point depends heavily on how expensive your analysis pipeline is. Heavy custom analyzers favor segment replication strongly; simple keyword indexing narrows the gap.

Trying It Out

Switching an index to segment replication is an index-level setting. On recent OpenSearch versions you can create a new index like this:

PUT /my-index
{
  "settings": {
    "index": {
      "replication": {
        "type": "SEGMENT"
      },
      "number_of_shards": 1,
      "number_of_replicas": 1
    }
  }
}

Existing indices keep their replication model, so adoption typically means new indices plus an index-pattern or ISM-policy change rather than an in-place flip. Pair the switch with ISM (Index State Management) if you run time-series data, so new rollover indices pick up the setting automatically.

After switching, watch these metrics: indexing throughput, CPU utilization per node, network outbound on primary-holding nodes, and replication lag. If segment replication is working for your workload, the first three will drop noticeably and the lag will stabilize at a small, predictable value.

The Bigger Picture

Segment replication is part of a broader trend in distributed search infrastructure: separating the compute-heavy work of building indexes from the serving work of querying them. Once replicas consume finished segments rather than rebuilding them, new possibilities open up: remote-backed primary storage, searchable snapshots without full restores, and compute that scales independently of storage. OpenSearch's investment in this model is really an investment in a more flexible architecture for the next decade of search workloads.

If you operate OpenSearch at any real scale, this is one of the highest-leverage settings to understand and tune. The difference between re-indexing every document on every replica and copying a segment file is the difference between paying for Nx indexing compute and paying for it once.

---

I'm Prithvi S, Staff Software Engineer at Cloudera and Opensource Enthusiast. I contribute to OpenSearch and related open-source search projects. Follow my work on GitHub: https://github.com/iprithv