DEV Community

Cover image for The integration is never the quickstart
William Taylor
William Taylor

Posted on

The integration is never the quickstart

Every third party service you have ever added to a product has a quickstart page that takes about eight minutes. Install the client library, paste a test key, call one method, see a response. The page is honest. It is also the smallest possible version of the work, and the gap between it and a shipped integration is where most delivery estimates quietly die.

The quickstart runs on the vendor's account, in their sandbox, on the happy path, with one request at a time, from a machine that is not under load and has nothing else to do. Production has none of those properties.

Here is what the remaining work actually consists of, roughly in the order it surprises people.

The sandbox is a different product

Sandboxes are built to be convenient, not accurate. They approve every card, verify every document, return every address, and they do it instantly.

Production declines things. Cards get declined for a dozen reasons that each need different copy on screen. Address lookups return nothing for valid addresses. Identity checks come back as "review" rather than yes or no, and nobody wrote a screen for review.

Before you estimate, ask the vendor for the list of every status their service can return, not the list in the quickstart. Then count how many of those statuses your product currently has a screen for. The difference is real work, and it is usually a week you did not plan.

The webhook is the integration

Most of these services are not request and response. You ask for something, they go away, and they call you back. That callback is where the integration actually lives, and it has properties the quickstart never mentions.

It can arrive twice. It can arrive out of order, so the completion lands before the creation. It can arrive while your server is restarting and be retried for a day, or not retried at all, depending on the vendor. It can arrive for an object you have never heard of because somebody made a refund inside the vendor's own dashboard.

The receiving endpoint therefore has to verify the signature, be safe to call repeatedly with the same event, tolerate events arriving in the wrong order, and record events it does not understand instead of failing on them. That is a day of careful work and another day of tests, and none of it is visible on a screen.

A rule that saves a lot of pain: the callback should write the event down and return success immediately, then do the real work from your own queue. Vendors retry when you are slow, and a slow handler turns one event into five.

Nobody owns the account

This one is not technical and it causes more delay than anything above.

Somebody has to create the production account, enter the company's legal and banking details, pass the vendor's review, and hold the credentials. In a client project that person is usually not on the engineering team, is often travelling, and has not been told that the launch date depends on them.

Ask on day one who that person is. Ask for the production credentials in week one even though they are not needed until week six, because the review on the vendor's side can take ten working days and nobody tells you that in advance.

While you are there, decide where the credentials live, who rotates them, and what happens when the person who created the account leaves the company. Every long running product eventually meets a key that nobody can regenerate.

Limits you will only find by hitting them

The vendor publishes a rate limit. What they do not publish is how your own code behaves when it meets that limit.

Most teams discover this during a migration or a backfill, when a loop that was fine for ten records meets ten thousand. The vendor starts refusing requests, the code treats the refusal as a failure, and half the batch is now in an unknown state.

Decide early what a refusal means. Usually it means wait and try again later, not fail. That implies the work has to be resumable, which implies you need to know which records are done, which implies a status column you would not otherwise have written.

The same applies to limits on the size of a request, the number of items in one call, and how many of your own requests can be in flight at once.

The failures you cannot reproduce

Your tests cover your own mistakes. They do not cover the vendor returning an internal error, timing out after thirty seconds, or returning a success with a body that does not match their own documentation. All three happen.

So decide, per call, what your product does when the vendor is simply unavailable. For a payment that usually means tell the user plainly and keep their basket. For a map or an address lookup it usually means let them type it manually. For analytics it means drop it silently, because nobody should fail a checkout over a tracking call.

Writing that decision down per integration takes an hour and removes a whole class of production incidents where a non essential service took down an essential flow.

Versions, and the email you will miss

The service you integrate today is a version. In eighteen months there will be a deprecation notice sent to the address of whoever created the account, who may no longer work there.

Pin the version explicitly rather than taking the default, keep the vendor's client library updated on a schedule instead of never, and put the integration's owner in writing. Products that break in year two usually break here, and the break is always discovered by a customer rather than by the team.

What this means for the estimate

A realistic shape for a non trivial integration is one day for the quickstart path, three to five days for the states, the callback handling and the failure behaviour, one to two days of testing against the real service with real credentials, and an unknown number of calendar days waiting for account approval that are not engineering days at all but still move the date.

That is why "add payments" is never a two day task, and why anyone who quotes it as one has read the quickstart and nothing else.

The useful habit is to estimate integrations separately from features and to state the waiting time as its own line, because it is the only part of the plan that no amount of engineering effort can shorten.

The short version

Read the vendor's list of statuses, not their quickstart. Treat the callback as the real integration and make it safe to receive the same event twice. Get the production account started in week one. Decide what every call does when the vendor is down. Pin the version and name an owner.

At Appagentix we build these into the plan as separate line items from the first week, across iOS, Android, React Native and Flutter, because the integration list is usually the most accurate predictor of how a delivery date will move.

Top comments (0)