A shipment event arrives six hours late. Another event reports that the same pallet has been received twice. Meanwhile, the warehouse management system (WMS) has already updated inventory, the enterprise resource planning (ERP) system has posted a receipt, and the transportation management system (TMS) still shows the shipment as in transit.
This is a common type of distributed-systems problem in supply chain software. The issue is not simply that one system received incorrect data. Multiple applications may process the same business event at different times, leaving the organization with conflicting records of what happened.
Supply chain management software development must account for delayed messages, duplicate notifications, out-of-order events, partial failures, and retries. Without those controls, a single shipment update can trigger duplicate inventory movements, incorrect available stock, inaccurate financial records, or unnecessary replenishment orders.
Why Shipment Events Arrive Late or More Than Once
A modern supply chain application rarely operates as one isolated system. Warehouse scanners, carrier APIs, ERP platforms, supplier portals, transportation systems, and inventory services exchange events through APIs, message queues, webhooks, or scheduled integrations.
These connections can fail in several ways.
A warehouse device may temporarily lose connectivity and upload its events after reconnecting. A carrier webhook may be retried because the receiving service did not acknowledge it in time. A message broker may redeliver a message after a consumer crashes before confirming processing.
Events can also arrive out of order. For example, a shipment's departure event may reach the integration service before an earlier loading event.
The receiving application cannot assume that every incoming message is new, current, or ready to apply.
Design Idempotent Event Processing
Idempotency means that processing the same event more than once does not produce additional unintended effects.
Suppose a warehouse receives 100 units of a product. The WMS sends a receipt event, but the acknowledgement is lost. When the sender retries, the receiving service must recognize the repeated event instead of increasing inventory by another 100 units.
A robust event-processing workflow should:
- Assign or retain a stable event identifier.
- Validate the source, event type, and required business identifiers.
- Check whether the event has already been processed.
- Apply the permitted state transition and inventory movement.
- Record the processing result.
- Acknowledge the event only after the required transaction is safely committed.
A deduplication table or equivalent durable event ledger can record processed event identifiers. Where possible, the deduplication record and inventory update should be committed in the same database transaction.
This matters because checking for duplicates in one transaction and updating stock in another can create a race condition. Two workers might both decide that an event is new before either records it as processed.
Do Not Confuse Duplicate Events With Duplicate Business Actions
Two events can describe the same business action without having identical payloads. A carrier might send one notification with a shipment status and another with an updated timestamp. Conversely, two legitimate receipt events may reference the same purchase order but represent different deliveries.
Therefore, deduplication should not rely only on matching product IDs, shipment IDs, or timestamps.
The system needs a stable event identity where available, plus business validation. A unique event ID can prevent a repeated message from being applied twice, while a separate business rule can prevent a shipment receipt from exceeding the quantity expected under a particular delivery or purchase order.
The goal is to eliminate repeated processing without suppressing legitimate movements.
Handle Out-of-Order Events With State Transitions
A shipment usually follows a series of business states: created, allocated, picked, packed, dispatched, in transit, delivered, and received. Actual workflows may contain exceptions, partial deliveries, returns, and cancellations.
A naive implementation might update the shipment status whenever a new message arrives. That approach can cause a late event to overwrite a more advanced state.
For example, a delayed IN_TRANSIT event should not automatically replace a confirmed DELIVERED state.
A safer design validates each transition against the current state and the source event's meaning. The system may use event sequence numbers, source timestamps, version numbers, or authoritative status queries where available. Timestamps alone are not always reliable because clocks can differ and events can be generated at different stages.
When an event cannot be safely applied, the system should preserve it for review or reconciliation rather than silently discarding it.
Keep Inventory Movements Separate From Shipment Status
A shipment status and an inventory movement are related, but they are not interchangeable.
A carrier reporting that a truck has arrived at a distribution center does not necessarily mean the warehouse has physically received and verified the goods. Similarly, a delivery confirmation may refer to the destination rather than a completed put-away operation.
Supply chain software should define which business event authorizes each inventory change.
For example:
- A dispatch confirmation may reduce the stock available at the shipping warehouse.
- A goods-receipt confirmation may increase received stock at the destination.
- A quality-inspection event may move units into a restricted or quarantine state.
- A put-away confirmation may move units from receiving to a storage location.
This separation prevents a shipment notification from incorrectly changing sellable inventory.
Use Durable Queues and Transactional Outbox Patterns
When one service updates a database and then publishes an event, a failure between those operations can leave the database updated but the message unsent.
The transactional outbox pattern addresses this by recording the business change and an outgoing event in the same database transaction. A separate publisher then sends the event to the message broker.
On the receiving side, an inbox or event ledger can track incoming messages and prevent repeated processing.
These patterns do not remove every failure. They make failures recoverable by ensuring that the system has durable records of the work that still needs to be delivered or processed.
The architecture should also define retry limits, backoff strategies, dead-letter handling, and procedures for replaying failed messages safely.
Reconcile Systems Instead of Assuming They Agree
Even with careful event processing, ERP, WMS, and TMS records can diverge because of configuration errors, integration outages, manual adjustments, or unsupported transitions.
A reconciliation process should compare the systems' business records using meaningful identifiers such as shipment ID, purchase order, receipt reference, SKU, location, and quantity.
Useful exception categories include:
- Shipment marked delivered but no warehouse receipt exists.
- A receipt exists in the WMS but not in the ERP.
- The same receipt reference appears more than once.
- Received quantity exceeds the expected delivery quantity.
- Inventory has moved into an unavailable or quarantine state unexpectedly.
- A shipment status is inconsistent with the documented event history.
Reconciliation should create actionable exceptions with traceable records. Automatically forcing two systems to match without understanding the discrepancy can hide the underlying problem.
Testing Failure Scenarios
A supply chain application should be tested against failure conditions, not just normal API responses.
The test plan should include duplicate webhooks, delayed events, out-of-order messages, worker crashes, database timeouts, message-broker outages, partial receipts, and manual inventory corrections.
For each scenario, verify that the system preserves the correct inventory balance, records the event outcome, and supports safe recovery. Load testing should also establish how the system behaves when a carrier or warehouse reconnects and sends a backlog of events.
Useful operational metrics include duplicate-event rates, processing latency, retry counts, dead-letter queue volume, reconciliation exceptions, and the time required to resolve an inconsistency.
Recommended Supply Chain Software Development Companies
Organizations implementing these controls may need custom event-processing services, ERP/WMS/TMS integration, or changes to an existing supply chain platform. Dev Technosys is one company to consider for custom supply chain management software development, particularly where business-specific workflows and system integrations need to be planned together.
Other options include enterprise technology providers and integration specialists whose experience matches the organization's existing systems. The selection process should verify event-processing design, transaction boundaries, observability, recovery procedures, and responsibility for ongoing integration support.
Conclusion
Late and duplicate shipment events are inevitable in distributed supply chain environments. Inventory corruption is not.
Idempotent processing, durable event records, validated state transitions, explicit inventory rules, and reconciliation make it possible to recover from delayed or repeated messages without creating additional business errors.
The strongest supply chain architecture treats event delivery as unreliable, records every meaningful transition, and makes uncertainty visible to operations teams. That is the foundation for keeping warehouse inventory, shipment status, and ERP records consistent as the business scales.
Top comments (1)
The distinction between a duplicate event and a duplicate business action is the part most dedupe designs miss — identical payloads are the easy case, while a carrier sending the same shipment status twice with different timestamps sails straight past a payload-hash check. Committing the dedupe record in the same transaction as the inventory update matters for the same reason: the check-then-update gap is exactly where two workers both decide an event is new. And the opening picture, three systems each holding a different truth about one pallet, is why this cannot be fixed at any single endpoint — the state transition rules have to be shared, or each system reconciles to a different answer.