Most system rollouts I've seen have a quiet second project hiding inside them. The ERP or CRM goes live, everyone celebrates, and about six weeks later somebody asks why orders from the website still get retyped by hand, why tracking numbers aren't flowing back from the carrier, and why the payment processor and the accounting system disagree about what got paid.
That second project is integration. It was never scoped, so it shows up late, unbudgeted, and urgent. In my experience it can cost about as much as the first project.
I'm Justin Pennington, founder of Infraxio, based in Ponte Vedra in St. Johns County, Florida, just south of Jacksonville. We build operations software for mid-size businesses. This is a builder-oriented rewrite of an essay on my site, The Integration Layer Nobody Plans For. More about me and what we're building: infraxio.com/justin-pennington.
Why integration keeps getting pushed to "phase two"
Integration gets deferred for two honest reasons. It's genuinely complex, and the core system feels like the real deliverable. Nobody demos a webhook handler.
But the core system never runs alone. The moment an order comes in from the storefront, a label gets printed by the carrier, or a payment clears at the processor, data has to cross a boundary. At that point integration is the critical path, whether or not it's on the project plan.
Draw the map before you write the first mapping
Before implementation starts, list every system that has to exchange data with the new platform. For each connection, write down four things:
- What data moves. Orders, customers, inventory counts, invoices, status changes.
- Which direction. One-way, two-way, and who wins when both sides change the same record.
- How often. Real-time events, a polling interval, or a nightly batch.
- What happens when it fails. Who finds out, how fast, and what the business does in the meantime.
That fourth column is the one most teams leave blank, and it's the one that decides whether the integration is an asset or a pager.
A plain table in the project doc is enough. The point isn't the format. The point is that every arrow between two boxes becomes a scoped, estimated piece of work instead of a surprise.
Point-to-point or a hub in the middle
There are two basic shapes.
Direct integrations have system A call system B's API. They're fast to build and easy to reason about when there are only two or three connections. The cost shows up as the count grows: every new system adds connections to several others, and each one carries its own auth, retry behavior, and quirks.
Middleware puts a central layer in the middle, whether that's a tool like Zapier or Make or a small integration service you own. Each system talks to the hub, and the hub routes and transforms. You get one place to watch, log, and change mappings. You also get one more system to run and keep healthy.
My rough rule: a couple of simple, one-way syncs can go direct. Once you're past a handful of systems, or the data needs real transformation along the way, a hub usually pays for itself in maintenance alone.
Error handling is the actual job
Making data flow on the happy path is the easy part. The real engineering is everything else:
- The other API is down. Queue the work and retry with backoff instead of dropping it.
- The payload is malformed. Reject it loudly, park it somewhere a human can see it, and don't half-write it.
- The record exists on one side but not the other. Decide in advance whether you create it, skip it, or flag it.
- The same event arrives twice. Make writes idempotent so a retry doesn't create a duplicate order or a double invoice.
- Nobody notices. Alert on failures and on silence. A sync that quietly stopped an hour ago is worse than one that errors.
If an integration doesn't have retries, a dead-letter path, and alerting, it isn't finished. It's a demo that happens to be in production.
What changes when you plan it up front
When the integration map is part of the original scope, a few good things happen. The budget is honest. Go-live means the business actually runs on the new system, not that the new system exists while people re-key data around it. And the team that owns the core platform also owns the boundaries, instead of handing them to whoever is free in month three.
That's the habit I push on every rollout we run at Infraxio here in Northeast Florida: draw the arrows first, then build the boxes.
Originally published on penningtonj.com. Justin Pennington, founder of Infraxio, Ponte Vedra.
Top comments (0)