DEV Community

yutianle
yutianle

Posted on

Your AI Gateway Is Holding the Keys: Hardening Self-Hosted LLM Infrastructure After the LiteLLM and Starlette KEV Additions

#cybersecurity #aiinfrastructure #authentication #llmsecurity

Your AI Gateway Is Holding the Keys: Hardening Self-Hosted LLM Infrastructure After the LiteLLM and Starlette KEV Additions

On 2 September 2026, CISA added seven vulnerabilities to its Known Exploited Vulnerabilities catalog in a single batch. Three of them target AI and ML infrastructure: CVE-2026-59822 in LiteLLM, CVE-2026-48710 in Starlette, and CVE-2026-82329 in JFrog Artifactory. It is the first time AI tooling has appeared in the catalog in this concentration, and it marks a shift that security teams have been slow to absorb. The gateway that brokers access to language models is now a credential store, and it is being attacked as one.

What the three flaws have in common

CVE-2026-59822 is an authentication bypass in LiteLLM, rated 8.8. The flaw lets an unauthenticated attacker reach the MCP session handling path, where an OAuth2 fallback can be replaced with an empty authentication object. In practice, the gateway stops asking who is calling.
CVE-2026-48710, rated 6.5, is a request smuggling and path-authorisation bypass in Starlette, the ASGI framework that underpins FastAPI. A malformed Host header shifts the path boundary, so an authorisation check evaluates one path while the application serves another. Because vLLM, LiteLLM, and many MCP servers are built on this stack, the flaw propagates far beyond the framework itself.
CVE-2026-82329, rated 9.8, is an authentication bypass in JFrog Artifactory that works in the default configuration. The Access component treats a blank join key as a trusted value, which allows an unauthenticated attacker to forge an HS256 JWT, call the unauthenticated cluster join endpoint, obtain a service token with admin scope, and exchange it for a platform administrator token.

Why AI gateways are a distinct problem

A traditional reverse proxy forwards traffic. An AI gateway holds secrets. LiteLLM and comparable tools store provider API keys for OpenAI, Anthropic, and others, they hold the credentials used to reach internal model endpoints, and they often log prompts and completions. An attacker who bypasses authentication on the gateway does not need to compromise the model provider. The gateway will make the call on their behalf, using keys the organisation pays for.
That is why the LiteLLM flaw is more than an access-control bug. It is a path to credential theft and to unauthorised consumption of metered services, both of which can go unnoticed for a long time because the traffic looks legitimate.

A hardening checklist

Patch all three components. LiteLLM, Starlette, and Artifactory have released fixed versions. Because the flaws are in the authentication layer, a version check is not sufficient; confirm the running build.
Inventory your AI infrastructure. Most organisations know they run a model API. Fewer can name every LiteLLM instance, every vLLM deployment, and every MCP server, or say which of them are reachable from the internet.
Remove internet exposure. An internal gateway does not need a public address. Where external access is required, put it behind an authenticated proxy with its own authorisation layer.
Rotate provider keys. If a gateway was exposed and unpatched, treat every provider key it stored as compromised. Check provider billing dashboards for consumption that does not match internal usage.
Enforce authentication at more than one layer. The LiteLLM flaw works because a single check can be bypassed. A second, independent check in front of the gateway turns a single point of failure into a defence in depth problem.
Log and alert on authentication anomalies. An empty or malformed authentication object reaching a session handler should be a detectable event, not a silent success.

The uncomfortable part

AI infrastructure was deployed quickly, often by teams measured on capability rather than on security posture. The result is a class of internet-reachable services that hold high-value credentials and were never designed for hostile exposure. The September KEV batch is the first clear signal that attackers have found them. It will not be the last.

References

Top comments (0)