What I Learned From Integrating 3rd Party APIs

I've worked with a lot of third-party APIs: telecom and messaging services, social media platforms, and a few others. This post is what I learned about keeping those integrations from breaking, in the order I ran into the problems.

I'm still not a pro, and I won't cover everything. This is just how I know it. I split it into three stages: novice, a little knowledge, and intermediate. Each stage starts with a problem the last one couldn't handle.

The short version

• Always set a timeout. Retry only errors that can go away on their own, with backoff and jitter. Know when to give up.
• Respect rate limits and 429 responses. Put work the user doesn't wait for on a queue.
• Make repeated operations safe. Then add circuit breakers, fallbacks and dead letter queues if you need them.
• Track latency, error rate and retries for each provider.
• Don't build all of it at once. I start with timeouts, rate limits, queues and retries.

Stage 1: novice

My first integrations were simple. I called the API over HTTP, wrapped it in a try/catch, and logged the error if it failed. Nothing much. My error handling was basically console.error and hope.

try {
  const res = await fetch("https://api.provider.example/v1/messages", {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify(payload),
  });
  if (!res.ok) throw new Error(`Provider returned ${res.status}`);
  return await res.json();
} catch (err) {
  console.error("provider call failed", err);
  throw err;
}

It all looked good at first. Then the first real problem showed up.

Stage 2: a little knowledge

Timeouts

The problem was my API waiting on the external API. If the provider is slow or never answers, the request just sits there. It holds a connection and memory. Enough of those and my service runs out of resources, even though the problem is on someone else's server.

That's how I learned about timeouts. You set one (30 seconds, or whatever fits your use case) and throw an error when it passes. If a user is waiting, use a shorter one. A background job can wait longer.

const res = await fetch(url, {
  method: "POST",
  body: JSON.stringify(payload),
  signal: AbortSignal.timeout(30_000),
});

At first I thought a timed-out request meant the provider had done nothing. But a timeout only means I stopped waiting. The provider may have done the work and just answered too slowly. This causes trouble later, in stage 3.

Retries, and which errors deserve one

Next, my API felt unreliable. A user sent a request, the provider had a bad moment, and the user got an error. So I learned about retries. My first version retried every error.

That doesn't work either. Some errors go away on their own and some don't.

A transient error can go away: a timeout, a dropped connection, a 502 or 503. A non-transient error fails the same way every time: a 400 for a bad payload, a 401 for a wrong API key, a 404 for something that doesn't exist. Retrying those fixes nothing. A 401 doesn't turn into a 200 because you asked nicely five times. It just wastes calls and delays the real error.

Retry the first kind. Fail fast on the second.

A transient error doesn't always mean it's safe to retry. If a POST times out, the provider may have already done the work, and a retry could do it twice. I'll come back to this in stage 3.

Exponential backoff

I also found out that retrying too fast makes things worse. If the provider is struggling, hitting it again right away adds more load. So you wait between attempts, and you wait longer each time: 1 second, then 2, then 4, then 8. That's exponential backoff.

Jitter

Backoff caused its own problem. Say a thousand requests fail at the same moment. They all wait 1 second and retry together. They fail together, wait 2 seconds, and retry together again. It's like everyone redialing the same busy number at the exact same second. The provider gets hit in waves. This is called the thundering herd problem. With no quiet moment to recover, a small problem can get a lot worse.

The fix is jitter. Add some randomness to the wait so the retries spread out.

When to stop retrying

At first I didn't know you could drop a request. I thought I had to keep retrying until it worked, so my code was a loop with no exit.

Retrying forever is a bug of its own. The user is still waiting, your workers are busy, and the provider keeps getting hit. Drop the request when any of these is true:

• The error is non-transient. A 400, 401 or 404 fails every time, so drop it right away.
• You've tried enough times. Pick a number, like 4, and drop the request after that.
• You've waited too long. Give the whole request a time limit, like 10 seconds, no matter how many tries it takes. A 30 second timeout with 4 tries can leave the user waiting about two minutes. Nobody waits that long.
• The provider asks you to wait too long. If its Retry-After header says 5 minutes and you can only wait 10 seconds, drop it. Waiting won't help.

When you drop a request, still do something with it. If a user is waiting, tell them it failed and that they can try again. If it's background work, move the job to a dead letter queue (more on that below) so it isn't lost.

This code puts it all together: a timeout on every try, retries only for transient errors, backoff with jitter, the Retry-After header, and the rules for giving up.

class HttpError extends Error {
  constructor(
    public status: number,
    public retryAfterMs?: number,
  ) {
    super(`Provider returned ${status}`);
  }
}

// Timeouts, network failures, 408, 429 and some 5xx errors may be transient.
// Check your provider's docs. 400, 401, 404 and similar fail every time.
const TRANSIENT_STATUS = [408, 429, 500, 502, 503, 504];

function isTransient(err: unknown): boolean {
  if (err instanceof HttpError) return TRANSIENT_STATUS.includes(err.status);
  // fetch throws a TypeError on network failures, but a bug in your code can too
  return err instanceof Error && (err.name === "TimeoutError" || err.name === "TypeError");
}

const sleep = (ms: number) => new Promise((resolve) => setTimeout(resolve, ms));

async function withRetry<T>(
  fn: (timeoutMs: number) => Promise<T>,
  maxAttempts = 4,
  budgetMs = 10_000,
): Promise<T> {
  const deadline = Date.now() + budgetMs;
  let lastErr: unknown;
  for (let attempt = 1; ; attempt++) {
    const timeLeft = deadline - Date.now();
    if (timeLeft <= 0) throw lastErr ?? new Error("Out of time"); // don't start another try
    try {
      // each try gets at most 3 seconds, and never more than the time left
      return await fn(Math.min(3_000, timeLeft));
    } catch (err) {
      lastErr = err;
      const backoff = Math.random() * 1000 * 2 ** (attempt - 1); // backoff + jitter
      const asked = err instanceof HttpError ? err.retryAfterMs : undefined; // from Retry-After
      const wait = asked ?? backoff;
      const outOfTime = Date.now() + wait >= deadline;
      if (attempt >= maxAttempts || outOfTime || !isTransient(err)) throw err; // drop it
      await sleep(wait);
    }
  }
}

const result = await withRetry(async (timeoutMs) => {
  const res = await fetch(url, {
    method: "POST",
    body: JSON.stringify(payload),
    signal: AbortSignal.timeout(timeoutMs),
  });
  if (!res.ok) {
    const seconds = Number(res.headers.get("Retry-After") ?? NaN);
    throw new HttpError(res.status, Number.isNaN(seconds) ? undefined : seconds * 1000);
  }
  return res.json();
});

This is a simple example to learn from, not code to paste straight into production. Change the number of tries, the waits and the time limit to fit your provider. Check its docs to see which errors it wants you to retry. Real code needs more care. For example, not every TypeError is a network problem, because a bug in your own code can throw one too. And POST requests are not always safe to retry (see stage 3).

Rate limits and 429

After that I ran into 429 Too Many Requests. It means I had sent more requests than the provider allows. That limit is called a rate limit. I retried harder, and it only made things worse. The provider was saying "slow down", and I answered by going faster.

So I learned about throttling. It means slowing yourself down, so you never go over the provider's limit. Use a limiter that caps how many calls you make each second or each minute. If you run more than one server, make them all share the same limiter. Otherwise each server counts on its own, and together they still go over the limit.

If a 429 still gets through, wait before you try again. Many providers send a Retry-After header. It tells you how long to wait. Use it instead of guessing.

Queues

Then I learned when to use queues. A lot of work doesn't need to happen while the user waits: sending emails or SMS, generating reports, and similar jobs. The request returns right away, and a worker calls the provider in the background.

This is where everything above came together for me. The worker handles the timeout, the retries with backoff and jitter, and the rate limit. You decide how many workers call the provider at once, so a spike of user requests doesn't become a spike of provider calls. If a job fails, the worker can retry it in the background. The user's request doesn't have to stay open while that happens.

Backpressure

A queue can also fill up. Say 1,000 SMS requests come in every second, but the provider only handles 100. The other 900 pile up in the queue, and jobs wait longer and longer. Backpressure means pushing back on new work when the rest of the system can't keep up. You can limit the queue size, slow down whatever adds jobs, or say no to new requests when you're full.

Stage 3: intermediate

Duplicate operations

Retries can cause a new problem: doing the same thing twice. Take the timeout from earlier. The provider may have already sent the SMS and just answered too slowly. If you retry, the user gets two SMS.

It can also happen without a timeout. Many queues promise to deliver a job at least once. That means the same job can run more than once. A worker sends the SMS, then crashes before it marks the job as done. The queue doesn't know the SMS went out, so it gives the job to another worker. That worker sends the same SMS again.

So the next thing I learned was how to make an operation safe to repeat. The word for this is idempotent: doing it twice gives the same result as doing it once. Whenever you can, make queue jobs idempotent, especially jobs that call an outside service or change important data.

If the provider supports idempotency keys, use them. An idempotency key is a unique ID that you send with each request. When the provider sees the same ID again, it knows the request is a repeat and doesn't do the work twice. Not every provider supports this.

If yours doesn't, keep your own record of what you've already sent. Check that record before you retry. And if you don't know whether the first try worked, ask the provider for the status first.

I wrote more about this in my post on payment integrations, where a duplicate means a duplicate charge.

Circuit breakers and fallbacks

This is where I got more excited about system design. I'm still new to this part. I only know the basics, and I'm learning more in my free time. Even a basic understanding of circuit breakers and distributed systems helps, though.

Say a provider is down for ten minutes. Retrying every request against it wastes time and ties up your workers.

A circuit breaker has three states. It starts closed, which means requests flow normally while it counts failures. When the failures pass a threshold, it opens, and requests fail right away without calling the provider. After a cool-down it goes half-open and lets a few test requests through. If they succeed, it closes again. If they fail, it opens again.

While the breaker is open, you need a plan B. That's a fallback. For SMS, it could be a second provider. For other things, it could be a saved answer from earlier, or a plain "try again later" message.

A retry tries again because the failure might be temporary. A circuit breaker stops trying because the provider looks unhealthy. They solve different problems and often work together.

Dead letter queues

Some jobs still fail after every retry. Retrying forever is a waste, and dropping them loses data. So they go to a dead letter queue. It holds them while you find out why they failed. Then you fix the cause and replay them.

Observability

Then someone asked me how my system was performing, and I had no idea. My answer was basically a shrug. That's when I learned about observability. I started with three things: latency, error rates and retries.

Look at latency in percentiles like p95, not just the average. The average hides the slow requests. Rising latency often shows up before the timeouts do.

The error rate tells you when a provider has gone bad, and which errors it returns.

Retries hide failures from users. So a rising retry count is how you spot a struggling provider before anyone complains.

You don't need all of it at once

All of this took me a lot of time to learn, so I wouldn't build it all in one go. Pick what fits the integration and the tradeoffs you're willing to take. A small script that calls one API once a day probably doesn't need a circuit breaker.

I usually start with timeouts, rate-limit handling, queues and retries. I add the rest when the integration needs it.

---

I'm Nishant, a backend-focused full-stack developer from Nepal. The original of this post, and the products I build, are on my site.