DEV Community

SmartGate profile image lizer yang
lizer yang for SmartGate

Posted on Originally published at smartgate.network

Enterprise AI Gateway Architecture Best Practices: MCP Gateway

#enterprise #ai #model #agents

Short answer: the MCP endpoint is the visible half of an enterprise gateway; the half that
enforces anything is the application around it. Eight decisions carry the weight: a tenant
provisioned with a plan and a token limit, an identity resolved per request, canonicalized
inputs before any comparison, policy read at runtime, prefix-scoped exclusions, validated tool
parameters, constant-time secret comparison, and plan refusals that arrive as typed errors.

Key takeaways

  • Limits are provisioned, not computed. newFreeTeamCreateData copies the plan's monthly token limit onto the team at creation, so the quota check always has a number to compare.
  • Identity and authorization are two steps, not one. getUserByEmail returns a principal or null and never throws, which keeps the 401-versus-403 decision in a single place.
  • Canonicalize before you compare. normalizePath gives one spelling per resource; if the writer and the checker disagree about spelling, a policy has a hole neither test can see.
  • Configuration is data with a closed default. getSitemapConfig reads both key lists in one query, normalizes every entry, and falls back to the smallest authority on error.
  • Scope is a subtree, not a substring. isPathExcluded is prefix-aware at the slash boundary, which is exactly the rule an agent memory namespace needs.
  • Validate shapes, reject loudly, compare secrets in constant time. hexToBytes throws on malformed input instead of zero-filling; timingSafeEqual reads every byte wherever the difference is.
  • Do this next: write down, on one page, what your gateway knows about a caller before its first tool call — tenant, plan, token limit, principal, and the scope it may reach. Whatever is missing from that page is missing from your enforcement.

Tenancy before traffic: what an enterprise AI gateway provisions first 

# lib/team.ts — source lines 17–30 (newFreeTeamCreateData)
function newFreeTeamCreateData(
  user: { name?: string | null; email?: string | null },
  userId: string,
) {
  const free = getPlanCatalogEntry("FREE");
  return {
    name: `${user.name || user.email || "User"}'s Team`,
    slug: slugify(user.name || user.email || "User"),
    plan: "FREE" as const,
    monthlyTokenLimit: free.monthlyTokenLimit,
    features: free.features as object,
    members: { create: { userId, role: "OWNER" as const } },
  };
}
Enter fullscreen mode Exit fullscreen mode

A gateway can only enforce a limit on something it has already described, so three facts have to
exist before the first tool call: an identity, a tenant and a plan. Here the tenant is written
with its plan and its monthly token limit in the same function that names it —
getPlanCatalogEntry("FREE") supplies the limit and the feature flags, and the creator is
inserted as an OWNER member in the same nested write.

That ordering is the decision. A limit provisioned at creation is data the quota check can
compare against; a limit computed on the first request leaves a state where nobody decided
anything, and the default reading of that state — no limit configured, so no ceiling — is the
one that appears on an invoice a month later. Reading the value from the plan catalog also makes
a plan change a migration of one row instead of a new branch in the enforcement path.

The failure mode this avoids is the common one: minting the agent's credentials before the
tenant exists. The key works, the tools run, and every audit row afterwards is unattributable to
a plan. Provision the tenant, write the limit, then issue the credential — that stored limit is
what the monthly check later reads, and the arithmetic of holding it per team is worked through
in enforcing a token quota per team.

The identity layer in an AI gateway architecture 

# lib/user.ts — source lines 3–19 (getUserByEmail)
export const getUserByEmail = async (email: string) => {
  try {
    const user = await prisma.user.findUnique({
      where: {
        email: email,
      },
      select: {
        name: true,
        emailVerified: true,
      },
    });

    return user;
  } catch {
    return null;
  }
};
Enter fullscreen mode Exit fullscreen mode

An AI gateway architecture has four separable jobs in its request path — parse the credential,
resolve the principal, authorize the action, enforce the limit — and the cleanest thing you can
do is keep them in different functions. This resolver is the second job: it returns the user
record, or null. It catches its own failure and hands the caller a value rather than an
exception.

The shape pays off twice. A lookup that returns a fact lets the transport layer own the status
codes, so "no principal" becomes 401 and "principal without membership" becomes 403 in one file
instead of being re-derived by every caller that catches a generic error. And a resolver with no
side effects is safe to call on any path, including the ones an audit has to describe after the
fact.

The query is scoped to two fields, name and emailVerified — least privilege in a select
clause is cheap, and it is the reason this function cannot leak a credential hash into a log
line. authentication for MCP traffic covers the credential side;
the architectural point is that OAuth tokens, API keys and a development bypass should resolve
to one internal principal, because per-key rate limiting and per-actor audit rows only compare
when the actor is the same kind of thing. Skipping the resolver — and letting a client assert
X-User-Id — moves an authorization decision from your gateway to your caller.

Canonicalize input before any comparison: the enterprise LLM gateway rule 

# lib/sitemap-config.ts — source lines 11–14 (normalizePath)
function normalizePath(path: string): string {
  let normalized = path.startsWith("/") ? path : `/${path}`;
  return normalized.replace(/\/+$/, "") || "/";
}
Enter fullscreen mode Exit fullscreen mode

Every enforcement decision in an enterprise LLM gateway is a comparison: this route against that
policy entry, this model alias against that allow-list, this tool name against that plan. A
comparison is only as good as the two strings, so canonicalization belongs at the boundary where
a value enters — request parsing, configuration load, policy save — and every internal
comparison works on canonical values.

The function here is four lines and makes three decisions. A path without a leading slash gets
one, so a resource has a single spelling. Trailing slashes are stripped, so /v1/models/ and
/v1/models cannot be two different resources in a policy table that lists only one of them. An
empty result collapses to / rather than the empty string, so "normalized to nothing" cannot
slip past a lookup.

One implementation is what prevents an asymmetry you cannot test your way out of. If policy is
stored normalized but requests are matched raw — or the reverse — a rule written for
/v1/models is bypassed by /v1/models/, and both functions are individually correct, so
neither unit test fails. The same bug appears wherever identifiers cross a boundary: aliases
differing by case or registry prefix, tool names carrying a namespace, slugs with a trailing
character. And normalizing is not validating — a canonical path can still be a forbidden one.
AI gateway vs API gateway is the same equality
question one level up: which hop a request goes through at all.

Configuration: what a Model Context Protocol gateway reads at runtime 

# lib/sitemap-config.ts — source lines 21–40 (getSitemapConfig)
async function getSitemapConfig(): Promise<SitemapConfig> {
  try {
    const configs = await prisma.systemConfig.findMany({
      where: {
        key: { in: ["SITEMAP_EXCLUDED_PATHS", "SITEMAP_ADDITIONAL_PATHS"] },
      },
    });
    return {
      excludedPaths: parseConfigList(
        configs.find((c) => c.key === "SITEMAP_EXCLUDED_PATHS")?.value,
      ).map(normalizePath),
      additionalPaths: parseConfigList(
        configs.find((c) => c.key === "SITEMAP_ADDITIONAL_PATHS")?.value,
      ).map(normalizePath),
    };
  } catch (err) {
    console.error("Failed to fetch sitemap config from DB:", err);
    return { excludedPaths: [], additionalPaths: [] };
  }
}
Enter fullscreen mode Exit fullscreen mode

Configuration is data, and the loader is where its shape gets fixed for everyone else. This one
reads two keys from a systemConfig table in one query, splits each stored string into a list,
normalizes every entry, and returns the excluded paths and the additional ones. One naming note before the code:
what the MCP acronym means is worth pinning down
first, because this loader reads configuration the protocol itself never defines.

Four properties transfer to a Model Context Protocol gateway's policy. One read for the whole
document: fetching each key on demand multiplies the round trips a request pays for and makes a
partially loaded policy possible. Parsing at the boundary: the stored string becomes a list
exactly once, so no consumer has to know the storage format. Normalization as it loads
(.map(normalizePath)): two consumers can never disagree about spelling.

The fourth is the interesting one. The error path returns empty lists, and that is conservative
only after you have decided which direction "empty" falls: for an allow-list, empty means deny,
which is the safe branch, while for a deny-list empty means allow everything, which is not. If
the same loader serves the control that decides which tools, models or memory namespaces are
reachable, store it as an allow-list, so a database error narrows the gateway instead of opening
it. Logging the failure, as this loader does, keeps that trade visible. The token half of the
same control plane is set out in
token optimization techniques, and the
placement of the gateway in the request path is where the gateway sits.

Scope, not just permission: what an agent memory may touch 

# lib/sitemap-config.ts — source lines 42–51 (isPathExcluded)
function isPathExcluded(path: string, excludedPaths: string[]): boolean {
  const normalized = normalizePath(path);
  return excludedPaths.some((ex) => {
    const normalizedEx = normalizePath(ex);
    return (
      normalized === normalizedEx ||
      normalized.startsWith(`${normalizedEx}/`)
    );
  });
}
Enter fullscreen mode Exit fullscreen mode

Permission answers "may this caller run this tool". Scope answers the harder question an agent
memory forces: which data may this run read, write or remember. The matcher here is one
function, and the line that matters is the boundary in its prefix test — a path is excluded when
it equals the excluded path or starts with the excluded path plus a slash.

Without that slash, an exclusion for /teams/acme would also exclude /teams/acme-prod and
every other path sharing the prefix. It is the classic prefix bug, and in a memory system it is
a cross-tenant bug. With the slash, an exclusion names a subtree — exactly the construction a
memory namespace needs: a run may write under teams/<id>/, and teams/<id>-staging/ is a
different subtree that has to be granted separately. Two agents sharing a prefix must never
share a store by accident, and the guarantee is a delimiter, not a string comparison.
Normalization happens inside the matcher, on both sides, which is why a rule from a settings
form and data from the memory tool can be compared at all — at the cost of O(n) work per check
over a list that only grows, so past a few hundred entries the first path segment wants to
become the index.

The last decision is the direction of the list. An exclusion list is a deny-list whose default
is "everything else is allowed", so every new namespace is born visible; for agent memory and
retrieval corpora, invert it into an allow-list of the namespaces a workflow may touch. Quota
bounds how much a run costs, scope bounds what it can reach — and only one of them can be
enforced by counting.
agent memory architecture covers what belongs in that store.

Input validation: where the MCP spec stops and the gateway starts 

# lib/crypto.ts — source lines 27–37 (hexToBytes)
function hexToBytes(hex: string): Uint8Array {
  const normalized = String(hex || "").trim().toLowerCase();
  if (!/^[0-9a-f]*$/.test(normalized) || normalized.length % 2 !== 0) {
    throw new Error("Invalid hex string");
  }
  const out = new Uint8Array(normalized.length / 2);
  for (let i = 0; i < out.length; i++) {
    out[i] = parseInt(normalized.slice(i * 2, i * 2 + 2), 16);
  }
  return out;
}
Enter fullscreen mode Exit fullscreen mode

The decoder here is eleven lines long and contains the three decisions every gateway's input
validation needs. It normalizes first — trim, lowercase. It validates the shape before it
allocates anything: a character-class test plus an even-length check, both of which reject
rather than repair. And it throws on a malformed value instead of returning a partially filled
buffer, because a decoder that silently zero-fills turns a caller's bug into a security event.

The MCP spec illustrates why this is the implementer's problem. It defines message shapes,
method names and a transport; it does not say what your server has to reject, how large a
payload may be, or which fields a plan bounds. Those are your decisions, and the cheapest place
to make them is the boundary where a tool's arguments arrive.

Three rules make that boundary earn its place. Validate once, at entry, and pass typed values
inward — validation repeated in two places drifts, and the drift is found by whichever caller
hits the other copy. Choose reject-or-coerce per field class: coerce enumerations and optional
filters so a typo narrows a request, and reject identifiers, hex values, keys and cursors,
because a typo there addresses a different object. And bound the expensive dimensions before any
work starts — text length, list depth, page size, steps in a chain — since an unbounded
parameter is a denial-of-service vector that costs tokens as well as latency. An error should
carry the offending field and the shape that arrived; the
MCP tools reference is where those shapes belong, so validation
and documentation come from one description.

Secret comparison: the MCP specification says bearer token, not how to compare it 

# lib/crypto.ts — source lines 39–48 (timingSafeEqual)
function timingSafeEqual(a: ArrayLike<number>, b: ArrayLike<number>): boolean {
  const lenA = a.length;
  const lenB = b.length;
  if (lenA !== lenB) return false;
  let diff = 0;
  for (let i = 0; i < lenA; i++) {
    diff |= ((a[i] as number) ^ (b[i] as number)) & 0xff;
  }
  return diff === 0;
}
Enter fullscreen mode Exit fullscreen mode

The MCP specification is explicit about the credential — an HTTP transport may act as an OAuth
2.1 resource server and accept bearer tokens — and silent about the comparison that follows.
That is the right division for a standard and a careless place for an implementation, because
the ordinary equality operator returns as soon as it finds a difference, and an attacker who
controls the input and can measure response time can recover a secret one byte at a time.

This function is the answer in nine lines. It checks length first, which reveals only the length
and is rarely a secret. Then it walks the whole buffer, XOR-accumulating each byte into one
integer and masking with & 0xff, and compares once at the end. The work is identical wherever
the first difference sits, and the result is a boolean, so no caller can branch on a partial
match.

Two details matter. The inputs are ArrayLike<number>, so the same helper verifies a
hex-decoded header — the decoder above produces exactly that — and an array of character codes:
one comparison primitive for every secret in the request path. And the length check is a
deliberate early return; the rule to keep is that content-dependent work stays branch-free. Its
place in a gateway is a short list: API-key comparison, webhook and HMAC signature verification,
and anything answering "is this the token you issued". Compare digests rather than raw secrets
when the secret is stored, so a timing leak exposes a hash. And keep the comparison on one code
path — a cache that returns early for known-good keys re-introduces the branch you just removed.
The contract this sits inside is walked through in
the specification walkthrough.

Plan gates: what a Model Context Protocol server returns when the plan says no 

# lib/exceptions.ts — source lines 2–4 (constructor)
constructor(message = "This action requires a pro plan") {
    super(message)
  }
Enter fullscreen mode Exit fullscreen mode

A plan gate is the gateway failure a client meets most often, and its shape decides whether an
operator fixes the problem in five minutes or files a ticket. The class here carries a default
message — this action requires a pro plan — so every raising site in a Model Context Protocol
server produces the same actionable sentence, and a subclass changes the message only to be more
specific. Three properties are worth requiring of the path.

It has to be typed. A distinct error class lets the request layer map the refusal onto a stable
code and a 4xx status. If plan refusals arrive as a generic 500, clients retry, the retries
count against the rate limit, and your dashboard shows an outage whose real cause is a
misconfigured plan. Typed errors are also what let an audit tell "asked and was refused" from
"crashed". And it has to name the way forward — the plan that would unlock the tool, the
capability involved, the plan the caller is on. "Forbidden" is not actionable; "smart_pipe is
available on Teams and above, and this team is on Free" is.

The third property is timing: the gate belongs at tool entry, before retrieval, before
compression, before a model call or a memory write, so a refused call costs nothing and leaves
no audit row that later reads as usage.
MCP logging and observability is where telling those two
rows apart becomes a habit rather than an incident review. Whether the tool list should also be filtered
per plan is the open question: hiding smart_pipe from a Free client's tools/list makes the
refusal rare, but the client has to refresh its cached list after an upgrade and the server-side
check stays necessary — the tool list is discovery, not enforcement.

The short version for whoever signs the invoice

Agents do not warn before they spend: an unsupervised loop calls tools, grows context and bills
until something external stops it. That is why the gateway, not the model, is the control point
a security review asks about — who called, what they were allowed to call, what it cost, and can
you prove it six weeks later.

SmartGate is an MCP-native algorithm gateway for token control, traffic shaping, and agent
audit. It is not a model host: you keep your model and your agents, each client gets one MCP
endpoint, and the gateway holds seven metered tools — smart_fetch, smart_search,
smart_context_gate, smart_dedup, smart_budget_guard, smart_memory and smart_pipe. The
plan ladder is the enforcement vocabulary rather than a price list: Free covers 2 million tokens
a month with 120 requests per minute per key and 7-day logs; Pro, from $18/month with the first
month at $5, raises that to 300 and 30 days; Teams, at $55/month, to 600 and 90 days; Enterprise
is contractual, with 200M+ pooled tokens, 1,200 requests per minute per key and 180 days.
Billing follows "Pay for the platform. Share only when you save." — the savings share begins
after $15 of measured savings, so the platform fee is the only line item you can forecast
(pricing).

What the standards already require

The MCP specification is deliberately thin here. Authorization is optional; where a transport
supports it, the server acts as an OAuth 2.1 resource server using bearer tokens
(MCP authorization),
and the transports page defines one Streamable HTTP endpoint with sessions optional
(MCP transports).
Nothing obliges a server to bound how often a client calls a tool, cap what those calls cost, or
keep a record.

The risk literature names the same gaps without closing them. OWASP's Top 10 for LLM
applications lists Excessive Agency (LLM06:2025) and Unbounded Consumption (LLM10:2025)
(OWASP LLM Top 10) — the tool-scope problem and the bill
problem, neither fixed by choosing a different model. NIST's AI Risk Management Framework frames
the work as govern, map, measure and manage
(NIST AI RMF). All three describe the
obligation and leave the mechanism to the implementer, which is why a review asks where
enforcement lives.

How SmartGate compares

The category splits by what sits in the request path and what it can stop. SmartGate's scope is
deliberately narrow: it governs tool traffic.

What it governs How you run it What you pay
SmartGate MCP tool calls: fetch, search, compression, dedup, budgets, memory, pipelines Hosted gateway; one MCP endpoint per client Free covers 2M tokens/mo with 120 req/min/key and 7-day logs; Pro from $18/mo (first month $5) (pricing)
LLM routers and proxies (e.g. LiteLLM, Portkey) Model calls: provider routing, keys, model-level spend Self-hosted or hosted, in front of the model API Usage-based on model traffic; tool calls sit outside their scope (LiteLLM, Portkey)
Cloudflare AI Gateway Model provider traffic: caching, analytics, provider limits Hosted at the edge, per provider route Free tier plus usage; a different hop from your tool calls (Cloudflare)
Kong AI Gateway Model and API traffic inside your own gateway estate Self-hosted; you operate and upgrade it Platform cost plus the engineering time to run it (Kong)
In-house middleware or MCP bridges Whatever you build, one capability at a time You own the runtime and the retries Engineering time; no budget, rate-limit or audit layer until you write it

If your problem is model routing, a router is the right tool and this gateway is not a
substitute for it. If agents call tools with credentials nobody tracks, a team can spend past
its plan, or nobody can reconstruct last month, those are tool-side concerns a router will not
answer.

How to get started

  1. Create the team and mint a key. Sign in at smartgate.network, create the team, and copy the key for the environment you are wiring — the plan and its token limit are written at that moment.
  2. Set the monthly token limit before connecting anything. The quota check falls back to the plan default when no explicit limit is configured; an explicit limit is what makes the exceeded flag mean something.
  3. Mount one client at a time. Point one agent at the Streamable HTTP endpoint, call a tool, and confirm the audit row carries the actor you expect. Running the gateway inside your own network changes the deployment contract but not the enforcement order — see the private-cloud deployment.
  4. Check the plan numbers against your workload. Compare expected requests per minute with your tier's per-key limit and the team ceiling above it — clients that share a key share its limit.
  5. Wrap loops with smart_budget_guard, and move multi-step work to smart_pipe. One audited trace per workflow is what keeps the log readable a month later.

Start on Free — 2 million tokens a month, all seven tools, 120 requests per minute per key, no
card: start free, then compare the caps
and retention windows your workload needs on the
pricing page.

FAQ

Is an MCP gateway the same thing as an AI gateway?
No, and the difference decides your architecture. Model gateways and routers manage provider
routing, keys and model spend; this gateway sits in front of tool calls and enforces per-key
rate limits, monthly token quotas, role checks and audit retention. Most enterprises want both
hops, and they are different buys.

Does the MCP specification require rate limits or audit logs?
No. Authorization is optional, and even where it is implemented it covers access tokens, not
call frequency or cost. Rate limits, quotas, budget caps and retention windows are
implementation choices, which is why a review asks where they are enforced.

What happens to rate limiting if Redis is unavailable?
In production the check fails closed: the request is denied with a short retry hint, because an
unmetered production gateway is worse than a throttled one. Outside production the same
condition returns an allowed response flagged as skipped, so local development stays usable.
Decide that branch explicitly — it is where availability and enforcement trade against each
other.

Can a plan limit be exceeded by configuring a policy that asks for more?
Not through the settings path. The clamp brings a stored policy inside the plan ceiling,
including a minimum daily cap a plan can impose, and forces plan-gated governance flags to
false. The assertion path is stricter: it raises instead of repairing, naming the field, the
limit and the plan.

How do I stop an agent loop from spending the month in one night?
With two limits of different shapes. The per-key rate limit bounds the burst; the monthly token
quota bounds the total, and the budget guard makes the same headroom visible to the agent so it
can stop voluntarily. Alerts alone do not stop a loop — enforcement does.

Which tools does the gateway expose?
Seven: smart_fetch for reading public URLs as Markdown, smart_search for multi-engine search,
smart_context_gate for compression, smart_dedup for overlapping passages, smart_budget_guard for
budget decisions, smart_memory for team-level memory, and smart_pipe for multi-step
orchestration. All seven are available on the Free plan.

Limitations and what this does not do

  • This is the application side of a gateway, not a network design. Subnets, TLS termination, secret distribution and backups are yours.
  • It governs tool traffic, not model traffic. Provider routing, model caching and prompt-level spend belong to a router or to the model provider.
  • Scope and quota answer different questions. A budget cap is not a sandbox: capping spend does not make an unsafe tool safe, and the gateway cannot judge whether a fetched page should have been fetched. OWASP's excessive-agency risk shrinks by scoping tools, not by metering them.
  • Limits are per key plus a team ceiling, not per agent role. Two agents sharing a key share its limit; independent budgets need independent keys.
  • Retention is an entitlement and the job prunes on schedule. Export before the window closes if your compliance requirement outlives the plan's window.

Sources

Method note

The code in this article is not transcribed. Each block was cut out of the slice body returned
by the SmartGate slice API and re-asserted byte-for-byte as a substring of that body before
publication; the first line inside every fence records the file and the exact source lines.
Symbols were pinned with whole-name containment (rule A level 2) and confirmed by the service's
slot-proof endpoint. One excerpt is three lines long: the plan-gate example is a constructor, so
its class and the site that raises it are described in prose rather than copied.

Slice provenance

# SERP keyword Symbol File Source lines How it was pinned sha256(12)
1 enterprise ai gateway newFreeTeamCreateData lib/team.ts 17–30 rule A L2 → slot-proof 4d810a8b9613
2 ai gateway architecture getUserByEmail lib/user.ts 3–19 rule A L2 → slot-proof d4be0d4cb56f
3 enterprise llm gateway normalizePath lib/sitemap-config.ts 11–14 rule A L2 → slot-proof 2eb5193f7ae1
4 model context protocol mcp getSitemapConfig lib/sitemap-config.ts 21–40 rule A L2 → slot-proof 181ec0c1fb02
5 agent memory isPathExcluded lib/sitemap-config.ts 42–51 rule A L2 → slot-proof 8f1268b57f71
6 mcp spec hexToBytes lib/crypto.ts 27–37 rule A L2 → slot-proof b52a5e2f7dd3
7 mcp specification timingSafeEqual lib/crypto.ts 39–48 rule A L2 → slot-proof d08fe27e2863
8 model context protocol server constructor lib/exceptions.ts 2–4 rule A L2 → slot-proof acd7732e5804

Every fenced block above was cut from the slice body and re-asserted against it byte-for-byte before
publication. 8 of 8 sections pinned, 0 abstentions, 0 misses.

Top comments (0)