From the team at Backboard.io. We build AI infrastructure, so we have a position here. We state it at the end, clearly labeled. Everything before that is just counting.
TL;DR
A typical production AI application runs on 6 to 9 separate vendors: a model gateway, a vector database, a memory service, a RAG framework, an embedding provider, an orchestration layer, an observability tool, and 2 to 3 model providers. Each vendor adds at least five things to your attack surface: a standing API key, an egress path out of your network, an SDK executing inside your runtime, a log store that fills up with prompts, and a subprocessor on your data processing agreement. Most teams never approved this stack as a whole. They approved it one sprint at a time. This post is about how to count what you have actually deployed, and what reducing it looks like.
What is an AI attack surface?
An AI attack surface is the sum of every point where an attacker could reach the data, credentials, or compute involved in your AI workloads. For a modern LLM application it has three parts: the model layer (the providers your prompts travel to), the data layer (every system that stores prompts, embeddings, memories, or retrieved documents), and the integration layer (every SDK, framework, and glue service with credentials to the other two).
The mistake most teams make is measuring only the first part. The model provider gets a security review. The seven services wrapped around it usually do not.
The stack you actually deployed
Here is the stack a typical team assembles for one production agent, and what each layer costs you in security terms.
| Layer | What it does | What it adds to your attack surface |
|---|---|---|
| Model gateway | Routes requests to providers | A standing key in CI, often with org-wide scope |
| Vector database | Stores embeddings for retrieval | A second queryable copy of your source data |
| Memory service | Persists user and agent state | Your prompts in someone else's logs, under their retention policy |
| RAG framework | Chunking, retrieval, prompt assembly | A large SDK and its dependency tree executing in your runtime |
| Embedding provider | Turns text into vectors | Separate billing, separate breach notification clock |
| Orchestration | Chains tools and agents | Glue code holding credentials for everything else |
| Observability | Traces and evaluates LLM calls | Prompt and completion logs leaving your boundary by design |
| Model providers (2 to 3) | Inference | Standing egress to each, each with its own retention terms |
Now multiply. Eight vendors means roughly eight API keys with their own rotation stories, eight egress destinations your firewall must allow, eight vendor security reviews (or eight reviews skipped), eight subprocessors added to your DPA, eight breach notification clocks that can start ticking independently, and eight dependency trees whose CVE feeds someone on your team should be watching.
None of these vendors is careless. That is not the point. The point is arithmetic: every additional system that holds a copy of your data or a credential to your systems is surface, no matter how well run it is.
Why vector databases are the layer nobody reviews
Vector databases deserve their own section because they carry the most misunderstood risk in the stack.
The common assumption is that embeddings are safe because they are "just numbers." The vector is treated as a one-way hash of the text. It is not. Embedding inversion is a published, reproducible attack class: Morris et al. (2023, "Text Embeddings Reveal (Almost) As Much As Text") demonstrated iterative reconstruction that exactly recovers 92% of short text inputs from their embeddings, and Song and Raghunathan (2020) showed embeddings leak both content and authorship. If your source text was sensitive, treat the vectors as sensitive. Full stop.
That reframes what a vector database is: a second, queryable copy of your source data, sitting in a different trust boundary, usually with its own API key, and frequently excluded from the data inventory your compliance team maintains.
Three questions to ask about yours today:
- Access: does the vector DB key in your app config have read access to every namespace, or is it scoped per workload?
- Inventory: does your data map list the vector store as a location where customer data lives? (If a regulator asks, "just numbers" is not an answer.)
- Deletion: when a customer invokes their right to erasure, does your pipeline delete the embeddings, or only the source rows? That third question is where most stacks fail, which brings us to the multiplication problem.
The deletion problem: one prompt, four retention policies
Follow one user message through a stitched stack. It lands in the gateway's request log. It is embedded, so a derivative lands in the vector database. The memory service persists a version of it. The observability tool captures the full trace, prompt and completion included.
That is one piece of customer data in four systems, under four retention policies, behind four vendor APIs with four different deletion semantics. When legal asks you to delete a specific user's data, you cannot execute that. You can file four tickets and ask nicely. Under GDPR Article 17 and similar regimes, "we asked our subprocessors" is a much weaker position than "we called the delete endpoint and logged the result."
Deletion you cannot prove is deletion you do not have.
The supply chain you inherited
The integration layer has its own history. LangChain shipped a remote code execution vulnerability in its math chain (CVE-2023-29374). The PyTorch nightly build was compromised through dependency confusion in December 2022. The OWASP Top 10 for LLM Applications lists supply chain vulnerabilities as a category precisely because the AI ecosystem moves fast and pins loosely.
Every framework you add is not one dependency. It is a tree. When your RAG framework has hundreds of transitive dependencies and executes in the same process that holds your database credentials, the framework's security posture is your security posture.
Shadow AI: the surface you did not approve at all
Everything above covers the stack you chose. There is also the stack you did not: employees pasting source code, contracts, and customer records into whatever consumer chatbot they prefer. Bans do not work; the incentive to use these tools is too strong. The pattern that does work is replacement: give people one sanctioned surface with access to the models they want, behind SSO, governed and logged. You cannot firewall your way out of shadow AI. You can only out-compete it.
How do you actually reduce an AI attack surface?
Vendor-neutral answer first. Four principles, in priority order:
- Minimize copies. Every system that holds prompts, embeddings, or memories is a copy. Fewer systems holding data beats more systems holding it well.
- Broker all egress. Model traffic should leave through one controlled point where you can enforce policy per workload, not through N SDKs each dialing their own provider.
- Consolidate the control plane, keep the keys. One place to rotate credentials, one audit trail answering "which model saw which data," with keys that remain yours (BYOK).
- Make deletion an API call. If you cannot demonstrate deletion across every copy programmatically, your retention policy is a hope, not a control. You can implement all four yourself with enough glue code and discipline. Some teams do.
Where we stand (the disclosed bias)
Backboard is the consolidated version of that stack: routing across 17,000+ models, memory (ranked first on the LoCoMo and LongMemEval benchmarks, receipts on GitHub), embeddings, agentic hybrid RAG, and stateful threads behind one API and one key. Enterprise deployments run as a signed container inside the customer's own cloud, behind the IAM, SOC, and audit infrastructure that already passed review. Egress to external models is opt-in per workload, never standing. Memory has full CRUD endpoints, so export and deletion are API calls you can put in a runbook.
The fair objection: "you just built a single point of compromise." Three answers. Consolidating the control plane is not consolidating the keys; BYOK means compromise of the platform does not hand over your credentials. In the enterprise deployment the one door is a door you already own, inside your own perimeter. And memory is exportable and the platform is model-agnostic, so consolidation does not mean lock-in.
You will also notice no certification badges in this post. That is deliberate. We publish no certification we have not earned. Ask for the architecture, not the badge.
FAQ
Are embeddings personal data under GDPR?
Treat them as if they are. Embedding inversion research shows vectors derived from personal data can be reconstructed into close approximations of the source text, which makes "anonymized because it is numeric" a hard position to defend.
Is a unified AI platform a single point of failure?
It concentrates the control plane, which is exactly what makes it auditable. The security question is where keys live and where data can go. With BYOK and deployment inside your own perimeter, the failure domain is one you already operate, instead of eight you do not.
How many vendors are in a typical production LLM stack?
Six to nine: gateway, vector database, memory, RAG framework, embeddings, orchestration, observability, and two to three model providers. Count yours by listing every AI-related line item in billing and every AI-related SDK in your lockfiles. The two lists rarely match, and the gap is unreviewed surface.
What is the fastest single improvement?
Egress. Route all model traffic through one brokered, policy-enforced path and turn off direct provider access from application code. It is the change with the highest ratio of risk removed to engineering effort.
Does consolidation hurt model choice?
It should not, and that is a test to apply to any platform: consolidation of the data plane and control plane is valuable, consolidation that locks you to one model is a different product. Insist on model-agnostic routing and exportable state.
Further reading
- Morris et al., Text Embeddings Reveal (Almost) As Much As Text (2023)
- Song and Raghunathan, Information Leakage in Embedding Models (2020)
- OWASP Top 10 for Large Language Model Applications
- NIST AI Risk Management Framework If you want to tear the argument down before you consider the product, the code is at github.com/Backboard-io. Questions and disagreements welcome in the comments.
Top comments (0)