By Demandify Media
A worker times out after sending a CRM update. The queue delivers the same message again. Did the first request fail, or did the response simply disappear? That distinction matters when lead enrichment can overwrite a verified field, trigger routing, or create a second task.
The useful engineering question is not whether retries happen. It is whether repeating an operation changes the record again. A repeatable enrichment pipeline needs an explicit update contract: what the request means, which evidence it carries, and when the system should decline to write.
Separate observation from permission to update
An enrichment response is a proposed observation, not an instruction to replace every matching CRM field. Store the returned value alongside its source, observation time, and matching basis before deciding whether it belongs in the operational record.
For example, a company employee band may describe the parent organization, while the contact works for a subsidiary. Retrying that observation should never make it more authoritative. Source precedence and identity checks must remain independent of delivery count.
Demandify Media's practical lead enrichment workflow provides the broader context for choosing useful fields and validating a match. The narrower concern here is preserving those decisions when an integration repeats work.
Define a stable operation identity
Give each proposed update a stable identifier that survives queue redelivery. Base it on the intended change, such as the CRM record identifier, enrichment request identifier, field name, and policy version. Include tenant scope when identifiers are only unique within an account.
Avoid generating a fresh random identifier inside every retry attempt. That makes duplicate deliveries appear unrelated. Also avoid using only the contact's email address: a later legitimate enrichment request must remain distinguishable from an earlier one.
Record processing outcomes durably. If an operation already completed, return its recorded outcome rather than applying the same side effects again. This is the practical purpose of idempotency: repetitions converge on one intended result. Decide how long those outcomes remain available, because deleting them before a delayed message returns can allow an old operation to run again.
Protect the write against a changed record
Deduplication alone cannot protect a field that someone corrected after the enrichment job began. Capture the relevant record version when preparing the proposed change, then use a conditional update if the CRM supports it.
When the version differs, reread the record and reevaluate the proposal. Do not silently force the old value over the new one. A recent human correction may deserve protection even when the enrichment source reports a newer observation date.
Where conditional writes are unavailable, document the weaker guarantee. A read followed by an ordinary write leaves a race window. Serializing changes through one worker can reduce competing writes, but another integration or user can still modify the record.
Keep side effects out of the retry loop
An accepted field update and a sales task are different operations. Treat them separately so a network timeout does not create another task or repeatedly move the same lead between owners.
One design records an accepted change and a pending downstream event together in the integration's own transactional store. A separate dispatcher then sends that event using its stable identifier. This does not magically make the CRM and local database one transaction; reconciliation is still necessary at their boundary.
For b2b lead enrichment, this distinction protects the handoff as well as the database. Teams connecting enrichment to demand generation and follow-up workflows need to know whether an owner assignment reflects a new decision or a repeated delivery.
Test failures between steps
Happy-path tests miss the moments that cause repeated writes. Use synthetic records and explicitly interrupt the integration at boundaries:
- After the CRM accepts a change but before the worker records success.
- After recording a downstream event but before dispatch finishes.
- While a user corrects the same field.
- When the provider returns an empty value during a retry.
Check the final field value, recorded outcome, and number of downstream actions. A successful HTTP response alone does not establish CRM data quality. Preserve the original verified value when an empty response carries no explicit deletion instruction.
Measure decisions separately from deliveries
Track delivery attempts, unique accepted changes, rejected proposals, and unresolved outcomes as separate counts. Otherwise an unstable connection can make activity appear to increase while no additional records improve.
Keep enriched attributes separate from engagement evidence, too. A repeated firmographic update should not add points each time a worker runs. The distinction between fit, behavior, and follow-up is explored in the lead scoring and nurturing guide.
Start with one field and prove that duplicate delivery, competing edits, and delayed responses produce understandable outcomes. Expand only after the contract is clear. For a discussion of how that contract fits a particular handoff, share the workflow context with Demandify Media.
Top comments (0)