DEV Community

Ivan Rossouw
Ivan Rossouw

Posted on

Counts Match. The Data Can Still Be Wrong

Many systems eventually need to repartition one persisted record. A batch becomes several processing groups. A container becomes several parcels. A work queue becomes several assignments.

The domain changes, but the risk is consistent: every child must move from one source into exactly one result without being invented, duplicated, or lost.

The tempting safeguard is a count comparison:

  • The source contains twelve items.
  • The proposed results contain twelve items altogether.
  • Therefore, the split must be valid.

Unfortunately, equal counts prove only cardinality. They do not prove identity.

The blind spot in a passing count

Imagine that a source contains the item IDs A, B, and C.

A proposed split contains:

  • Result one: A, A
  • Result two: C

The source and outputs both contain three entries. A count check passes, even though A was duplicated and B disappeared.

Another invalid proposal might contain A, B, and D. Again, the count matches, but an unknown item has replaced a valid one.

These are not arithmetic failures. They are failures of identity conservation.

State the stronger invariant

For a valid repartition, the original identity set must equal the disjoint union of the output identity sets.

That gives us three useful requirements:

  1. Every original ID appears in the outputs.
  2. No unknown ID appears in the outputs.
  3. No ID appears more than once.

Set equality proves membership. A duplicate check proves disjointness. Cardinality remains useful, but only as one part of the invariant.

Validate before mutation

A generalized C# guard can make that rule explicit:

static void EnsureExactPartition(
    IReadOnlyCollection<Guid> originalIds,
    IEnumerable<IReadOnlyCollection<Guid>> partitions)
{
    var original = originalIds.ToHashSet();
    var output = partitions.SelectMany(ids => ids).ToArray();

    var sourceHasDuplicates = original.Count != originalIds.Count;
    var outputHasDuplicates = output.Distinct().Count() != output.Length;

    if (sourceHasDuplicates ||
        outputHasDuplicates ||
        output.Length != original.Count ||
        !original.SetEquals(output))
    {
        throw new InvalidOperationException(
            "The requested split is not an exact partition.");
    }
}
Enter fullscreen mode Exit fullscreen mode

This example is deliberately small. In a real application, authorization, lifecycle state, concurrency, and ownership rules may add further checks. The important ordering is that the complete proposal is validated before tracked entities or database state are changed.

That ordering makes failure cheap. An invalid request becomes a rejected request, not a cleanup exercise.

Move tracked children instead of copying them

When the children already exist as tracked entities, repartitioning should preserve their identities.

Creating new child objects by copying fields can accidentally turn a move into duplication. It may also lose historical references, introduce new primary keys, or leave the originals attached to the source.

Instead, load the authorized source and its children, resolve each requested ID to the existing tracked instance, and re-parent that instance to its destination. With EF Core, this usually means updating the relationship through the navigation property or foreign key and allowing relationship fix-up to track the move.

The output records are new. The children are not.

Make the whole transition atomic

A valid split still should not become partially visible.

Creating two results, failing on the third, and leaving the source marked as processed produces a state that is difficult to reason about and harder to retry safely.

Treat these changes as one unit:

  • Create every result.
  • Move every child.
  • Transition the source record.
  • Save the complete state atomically.

A single SaveChanges call is transactional for supported relational providers. If the workflow requires multiple saves or coordinates additional durable work, use an explicit transaction or an outbox-style boundary.

The desired outcome is simple: observers see either the original state or the complete repartitioned state, never a mixture.

Give every result an idempotency scope

Operation-level idempotency is helpful, but a split produces several distinct results. Each result therefore needs a stable scope of its own.

A compound uniqueness boundary such as (operationId, stableResultKey) lets a retry identify each intended result independently. The result key should come from stable input or deterministic content, not an array position whose meaning may change when ordering changes.

A database uniqueness constraint should enforce this promise. Application checks improve error messages; the constraint protects correctness under concurrency.

The trade-off is intentional friction

Exact set validation allocates collections. Transactions hold resources. Unique indexes add storage and write cost. The implementation is more involved than comparing two integers.

For very large partitions, validation may need to move closer to the database or use streaming and batching. That changes the mechanism, not the invariant.

The additional friction buys something valuable: failures occur before mutation, concurrent retries converge, and the persisted model remains explainable.

Tests that expose the boundary

A compact test suite should include:

  • A valid partition in a different order
  • One duplicated ID
  • One omitted ID
  • One foreign ID
  • A retry using the same per-result scopes

The negative cases should fail before any entity is re-parented or source state is changed.

The practical lesson is modest: whenever one persisted record becomes many, ask more than, “Did the counts match?”

Ask, “Did every identity move exactly once?”

Top comments (0)