Grouped Batch Sync: Reducing Orphan Records in Offline-First Applications

Emerson Barcellos — Wed, 03 Jun 2026 20:52:59 +0000

Grouped Batch Sync: Reducing Orphan Records in Offline-First Applications

Offline-first synchronization sounds simple until one user action creates several related records.

Saving data locally is the easy part. The hard part starts later, when the application needs to synchronize that local state with the cloud and with other devices. If every record is synchronized independently, a business operation that was complete on one device can become incomplete on another.

That is how orphan records appear.

A receipt may arrive without its payment. A financial entry may point to a receipt that has not arrived yet. Inventory may be updated without the sale or receipt that explains the movement. The local device looks correct, but the rest of the system sees only part of the operation.

This article describes a practical technique I call Grouped Batch Sync: an offline-first synchronization strategy that combines local queues, operation groups, and technical batches without allowing a batch limit to split a business operation.

The Problem With Record-by-Record Sync

Many offline-first systems start with a local queue:

Save data locally.
Add a pending item to the sync queue.
Send pending items when the network is available.
Remove accepted items from the queue.

This works well when one user action maps to one record.

It becomes fragile when one user action maps to multiple records.

For example, saving a receipt can create:

a receipt record;
a payment record;
a financial entry;
inventory movements.

If each record enters the sync queue as an independent item, they may be sent in different requests, accepted at different times, or rejected independently. Another device may observe the operation halfway through.

That is not just a visual bug. It can create real business inconsistency.

The Core Idea

Grouped Batch Sync adds one concept to the local queue: an operation group.

When a user action starts, the application creates a syncGroupId. Every record generated by that action carries the same group identifier.

The sync engine still sends technical batches, but the orchestrator preserves logical groups inside those batches.

In other words:

Technical batch size should optimize transport. It should not be allowed to break a business operation.

Technical Batch vs. Logical Group

These two concepts are related, but they solve different problems.

A technical batch exists for performance:

send at most N records at a time;
avoid large payloads;
reduce timeouts;
protect the API from oversized requests.

A logical group exists for consistency:

keep receipt, payment, and financial entry together;
keep order and items together;
keep deletion and reversal records together.

Grouped Batch Sync combines both. It sends records in batches, but it treats records with the same syncGroupId as a unit when building those batches.

A Minimal Example

Imagine this local operation:

await SyncOperationContext.run(
  groupId: SyncOperationContext.createGroupId(
    type: 'receipt-create',
    rootId: 'receipt-001',
  ),
  groupType: 'receipt-create',
  action: () async {
    queue.upsert('receipts', 'receipt-001', {'total': 120.0});
    queue.upsert('payments', 'payment-001', {
      'receiptId': 'receipt-001',
      'amount': 120.0,
    });
    queue.upsert('financial_entries', 'entry-001', {
      'reference': 'receipt-001',
      'amount': 120.0,
    });
  },
);

Those three records are independent records, but they belong to the same business operation.

Now add two unrelated records:

queue.upsert('products', 'product-001', {'name': 'Paper roll'});
queue.upsert('customers', 'customer-001', {'name': 'Ada'});

Then flush the queue with a technical batch size of 2:

await orchestrator.flushPending(batchSize: 2);

A normal batching algorithm might send:

Batch 1:
- receipts:receipt-001
- payments:payment-001

Batch 2:
- financial_entries:entry-001
- products:product-001

Batch 3:
- customers:customer-001

That splits the receipt operation.

Grouped Batch Sync instead produces:

Batch 1 (3 records)
- receipts:receipt-001 action=upsert group=receipt-create / receipt-create:receipt-001:...
- payments:payment-001 action=upsert group=receipt-create / receipt-create:receipt-001:...
- financial_entries:entry-001 action=upsert group=receipt-create / receipt-create:receipt-001:...

Batch 2 (2 records)
- products:product-001 action=upsert group=no-group
- customers:customer-001 action=upsert group=no-group

Even with batchSize: 2, the first batch contains 3 records because they belong to the same logical operation.

This is the key behavior.

The Orchestrator Rule

The batching algorithm is simple:

for each pending item:
  if item was already consumed:
    continue

  if item has syncGroupId:
    nextItems = all pending items with the same syncGroupId
  else:
    nextItems = only this item

  if nextItems do not fit in the current batch:
    flush current batch

  add nextItems to current batch
  mark nextItems as consumed

flush last batch
send each batch to the API

This means a logical group can exceed the technical batch size. That tradeoff is intentional. The batch size is a transport optimization; the operation group represents business consistency.

In a production system, you may still want a maximum group size. But that should be a separate rule with explicit handling, not an accidental split caused by generic batching.

What This Solves

Grouped Batch Sync helps reduce:

orphan records;
incomplete states across devices;
partial visibility of composite operations;
difficult-to-debug sync failures;
cases where one device looks correct and another sees only half of the operation.

It is especially useful when an offline-first app has actions that create side effects across several entities.

What This Does Not Solve

This is not a universal solution for every offline-first problem.

You still need:

conflict detection;
conflict resolution policies;
record versioning;
idempotent server endpoints;
server-side validation;
retry handling;
deletion semantics;
observability for rejected records;
transactions if you need true atomic behavior on the server.

Grouped Batch Sync does not magically make distributed systems simple. It solves a narrower but important problem: preserving the shape of a composite business operation while it moves through an offline-first sync pipeline.

Implementation Notes

The method can be implemented with a few small pieces:

An operation context that stores the active syncGroupId.
A queue layer that attaches the group to every local change created inside that context.
A queue item model that stores syncGroupId and syncGroupType.
A sync orchestrator that chunks pending items while preserving groups.
A server endpoint that receives batches and applies validation consistently.

In Dart, an operation context can be implemented with Zone, which keeps contextual values available across asynchronous calls. Other ecosystems can implement the same idea with async-local storage, coroutine context, request context, or explicit parameters.

The concept is not tied to Dart or Flutter. The pattern is portable.

Why This Matters

Offline-first applications are becoming more common, especially in mobile, field operations, point-of-sale systems, local-first tools, and multi-device business apps.

But offline-first is not only about "saving while offline." It is about preserving meaning while data moves through time, devices, networks, retries, and conflicts.

When a user performs one business action, the system should try to preserve that action as a coherent unit.

Grouped Batch Sync is one way to do that.

Conclusion

The mistake is letting transport batching define business boundaries.

Technical batches are useful. They protect performance and reliability. But when a technical batch splits a logical operation, the sync system can create orphan records and incomplete states.

Grouped Batch Sync keeps these concerns separate:

batches optimize transport;
groups preserve business meaning.

That distinction is small, but in real offline-first applications, it can remove a lot of pain.

Repository

The example and technical documentation are available here:

https://github.com/emersonbarcellos/grouped-batch-sync

DEV Community: Emerson Barcellos

Grouped Batch Sync: Reducing Orphan Records in Offline-First Applications

Grouped Batch Sync: Reducing Orphan Records in Offline-First Applications

The Problem With Record-by-Record Sync

The Core Idea

Technical Batch vs. Logical Group

A Minimal Example

The Orchestrator Rule

What This Solves

What This Does Not Solve

Implementation Notes

Why This Matters

Conclusion

Repository