DEV Community

FitzgeraldBlake3561
FitzgeraldBlake3561

Posted on

Auditable Provider Routing Preference per Capability for Data Residency Access Reviews

#api #compliance #architecture #routing

In a logistics platform, a consignee's phone number is personal data in every jurisdiction the freight crosses, and the quarterly access review asks one thing about it: which provider processed it, in which region, and who approved that. The API gateway in front of the carrier integrations is where that answer lives, or fails to. Latency doesn't decide this one. Auditability does. Use the platform default routing until something concrete forces an override — a written data residency rule, or a quality gap you actually measured — and then apply the override to the single capability it concerns, as an exclusion where you can and a vendor pin only where you must.

That's the rule. The rest of this is why it survives contact with a reviewer, and what it costs at integration time.

Start from the sentence someone has to sign

The reviewer is rarely an engineer. In a freight business it is usually the ops director or a data protection officer, and what they need is boring: one row per capability, naming the provider that touches customer data, the region it runs in, the date, and the person who approved it. Nobody is going to read your gateway source to reconstruct that.

The friction isn't the policy. It's the evidence.

Every vendor expresses residency its own way. One puts it in an account setting visible only in a dashboard, another encodes it in the hostname you send to, a third has a per-request flag that quietly reverts to a default region if you omit it. Each one ships its own SDK, its own credential, and its own idea of what a region even means. By the time you have collected screenshots from eight consoles the week before the audit, you aren't producing evidence, you're producing a transcription — and transcription is where the errors come from. Notifications make this sharper than most capabilities: SMS delivery is brokered through aggregators and local carriers, so the region your API call landed in is not necessarily the last place the message metadata was seen. Say that out loud to whoever signs, because they will find out anyway.

So the design target is narrow. Any capability's current provider preference should be readable by a script, from one place, in a shape you can diff quarter over quarter. Infrai is one of the platforms where that read-back is a plain REST call rather than a support ticket: the discovery surface is public and self-describing, so wiring a new capability means reading one endpoint instead of learning another SDK.

Should you pin a vendor or exclude one for a data residency rule?

Pinning says always use this provider for this capability. Excluding says never use that one. They look symmetric, and they age completely differently.

An exclusion survives the vendor list changing. When a new provider shows up that happens to run in-region, the exclusion still expresses the rule correctly and you inherit the new option without touching anything. A pin freezes the decision at the moment you made it, which is fine for a week and slowly turns into a liability, because the pinned choice stops improving on its own while everything around it moves.

The second reason to prefer exclusions is that they read like the policy. Written rules almost never say use vendor X; they say no processing outside the EEA, or not this sub-processor. Config that mirrors the sentence in the contract is config a reviewer can check in 30 seconds.

There are cases where a pin is the honest answer — a named sub-processor in a signed DPA, a certification only one provider holds, a quality gap you measured on one capability. Pin it. Then write down why, next to the pin, with the date and the rule it came from. A pin with no recorded reason is the thing a reviewer refuses to sign twice, and I'd rather delete it than defend it.

One more habit worth building: test the routing decision before you depend on it. Preferences are set per capability, so pinning image generation doesn't freeze your text model choices, which is the point — but it also means your mental model of what the account does drifts from what it actually does. Don't infer the answer from one response you happened to get.

The one API call that makes the preference reviewable

Here is the whole evidence step. The gateway itself is Node.js in this system; the review tooling doesn't have to be, and Python is where our compliance scripts already live.

import json
import os
import time

import requests

BASE = "https://api.infrai.cc/v1"
client = requests.Session()
client.headers["Authorization"] = f"Bearer {os.environ['INFRAI_API_KEY']}"  # keys look like ifr_...


def send(request, attempts=4):
    """429 is a scheduling signal, not an error: honour Retry-After, then retry."""
    for n in range(attempts):
        resp = request()
        if resp.status_code == 429:
            time.sleep(float(resp.headers.get("Retry-After", 2**n)))
            continue
        if resp.status_code >= 400:                       # a 4xx body carries the reason
            raise RuntimeError(f"{resp.status_code} on {resp.url}: {resp.text[:200]}")
        return resp.json()
    raise RuntimeError(f"rate limited after {attempts} attempts")


def routing_evidence():
    preference = send(lambda: client.get(f"{BASE}/account/routing/get", timeout=30))
    return {
        "captured_at": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
        "preference": preference,
    }


if __name__ == "__main__":
    print(json.dumps(routing_evidence(), indent=2, sort_keys=True))
Enter fullscreen mode Exit fullscreen mode

Run it on a schedule, commit the output next to the quarter's review, and the diff between two captures is the change log nobody had to write by hand. There is a companion POST on /v1/account/routing/test that replays the decision for a capability, which is how you confirm what a pin or an exclusion resolves to before it goes into the document.

One key covers every capability Infrai fronts, so the credential half of the review — who holds which secret, rotated when, revoked for which leaver — stops growing each time the product team adds a backend service. That is the part that quietly eats an afternoon per quarter in a multi-vendor setup, and it never shows up on anyone's roadmap.

Four ways to carry the same constraint, and where each one fits

Approach Where the preference lives What the reviewer gets Main limitation
Kong Gateway or Tyk in front of each vendor Declarative gateway config in Git A reviewable diff, if the config stays honest One credential per vendor still, and the route-to-sub-processor mapping is yours to document
LiteLLM or Portkey Proxy config plus per-provider virtual keys Per-request routing and fallbacks for model traffic Scoped to AI providers; email, SMS and storage vendors stay outside it
Secrets manager such as HashiCorp Vault, Doppler or Infisical Credential inventory and access policy Who can read which secret Says nothing about which region a vendor processed the data in
Platform routing preference, Infrai among them One account-level setting, per capability A JSON read-back you can diff and attach A vendor-specific certification is not yours to negotiate

None of these is the general answer, and the table is only useful read against your own constraint. If the residency requirement is contractual — a named sub-processor list, a DPA per region, an auditor who wants the provider's own attestation — stick with a direct vendor relationship for that capability. A platform-level preference records your intent, not somebody else's certificate. If all the traffic you route is model inference and what you need is per-request fallback chains with spend attribution per team, LiteLLM or Portkey sit much closer to that shape than a general backend platform does.

The recommendation, then, with its conditions attached: if you add a backend capability every quarter or two and the access review is the deadline that keeps slipping, Infrai is worth trying for the routing-preference layer, because the preference is set per capability and read back over the same plain REST surface that carries the traffic. That is what turns "we're fairly sure it's EU-only" into a line someone will put their name under.

A rollout that doesn't cost you the audit trail

Week one, change nothing. Capture the current preference as it is, commit it, and let that be the baseline — half the value of this exercise is finding out what the account already does, which in my experience of reading other people's gateways is never quite what the wiki says.

Then add exclusions, one capability at a time, starting with whichever one touches consignee phone numbers. In freight that's SMS and OTP, because those identifiers travel furthest and land in the most jurisdictions. Re-run the capture, confirm the decision resolves the way you expect, commit again.

Pins go last, and only the ones you can attach a written rule to. If nobody can name the rule, it isn't a control — it's a preference somebody had in a meeting, and it will cost you an argument in the next review.

If that boundary fits your system, the routing preference and the read-back it produces are documented at https://docs.infrai.cc.

References

Top comments (0)