Originally published on CoreProse KB-incidents
From Product Darling to Incident Report: What Happened
Lovable Vibe was a “lovable” AI coding assistant inside IDE-like workflows.
It powered:
- Autocomplete, refactors, code reviews
- Chat over entire repositories
- All backed by a shared large language model (LLM) service
That meant routine access to:
- Source code and internal libraries
- Git credentials and configs
- PII pasted into prompts—turning the LLM layer into a high-value attack surface when wired into internal systems.[2][3]
Over 48 days, prompts, partial code, and chat histories were exposed across tenants.[3] This was a class of LLM data leak where sensitive data crosses boundaries during normal use—not a classic hacked database.[3]
⚠️ Key point:
This was a logical isolation failure in the LLM serving layer. A shared performance optimization (multi-tenant KV-cache) bridged tenants—similar to emerging side-channel risks in multi-tenant LLM and AI agent systems.[6][7]
As enterprises push AI into external apps and agentic workflows (Category 3–4 maturity), these failures become more damaging.[5] Perimeter tools (firewalls, WAFs) do not see prompt-level cross-tenant leakage driven by:
- Non-deterministic model behavior
- Shared KV-caches
- Internal scheduling logic[5]
Business impact:
- Customers feared their proprietary code and prompts were visible to others
- Confidence in Lovable Vibe’s isolation and compliance collapsed
- Rollouts were paused and migration plans started overnight.[3][4]
💼 Takeaway for engineering leaders
The rest of this article explains how multi-tenant KV-cache optimizations leak prompts and how to redesign architecture, code, and MLSecOps to avoid the same trust crisis.
Inside the Blast Radius: Multi-Tenant LLM Serving and KV-Cache Risks
Modern LLM serving stacks aggressively optimize cost by sharing a Key-Value (KV) cache across requests. Frameworks like vLLM and SGLang:
- Reuse attention KV states for identical token prefixes
- Reduce computation and GPU memory
- Are standard in high-throughput, multi-tenant setups[7][8]
Research shows these shared caches are potent side channels. Under scheduling like Longest Prefix Match (LPM), an attacker can infer other users’ prompts by probing the cache and measuring Time to First Token (TTFT) or response ordering—PromptPeek-style attacks.[6][7]
📊 PromptPeek-style attack in practice[6][7][9]
An attacker can:
- Train a local LLM on the target domain to guess next tokens
- Send batched queries differing only in the last token, padded with dummy tokens
- Observe which query is prioritized (TTFT or position in batch)
- Confirm the hit as the victim’s next token and iterate
With reinforcement-optimized local models, these attacks become efficient enough for real-world prompt reconstruction.[6]
Other work shows KV-cache sharing supports:
- Direct inversion and collision-based reconstruction
- Semantic injection and prompt injection
- Demonstrating how performance optimizations can turn into privacy leaks when not scoped by tenant/security domain.[8]
A plausible Lovable Vibe root cause: KV-cache entries and scheduling keyed only on token sequences, not <tenant, user, session, prefix>.[6][7] This allows:
- Cross-tenant cache reuse
- Latency-based inference of other tenants’ prompts
- In extreme cases, mixed prompts and responses
This illustrates that LLMs create a distinct attack surface, where adversaries exploit:
- Model behavior
- Intermediate representations (KV states, embeddings)
- Shared serving infrastructure—not just OS/network bugs[2][4]
Yet >65% of organizations running ML in production lack ML-specific security strategies, so such flaws reach users undetected.[4] Agentic AI with broad internal access amplifies the impact.
⚠️ Blast radius summary
A globally shared KV-cache without strong isolation lets any tenant with enough traffic and basic latency metrics infer or reconstruct others’ prompts, code, or PII.[3][6][8]
Threat Modeling the Lovable Vibe Incident: Adversaries, Vectors, and Data Types
Defenses require a concrete threat model for multi-tenant coding assistants, not generic “data breach” language.
Attacker profiles
Likely adversaries:
- Malicious tenant seeking competitors’ code or prompts
- Curious insider with access to logs, metrics, or scheduler internals
- Opportunistic attacker combining LLM-specific exploits (prompt injection, data poisoning) with standard weaknesses (misconfigured observability, exposed metrics endpoints)[2][4]
Organizations already report LLM abuse via:
- Prompt injection and data exfiltration
- Jailbreaking and malicious code generation
- Misuse of plugins/tools linked to internal APIs and DBs[2][4]
Attack vectors in this case
KV-cache–related vectors relevant to Lovable Vibe:
- Side channels (PromptPeek-like probing via TTFT and ordering)[6][7][8]
- Cross-tenant prompt/response interleaving from mis-scoped caches
- Prompt injection where one tenant’s prompt alters shared model state reused in others’ sessions[6][7][8]
💡 Example
An engineer at a 30-person SaaS startup noticed autocomplete suggestions containing variable names and function headers from unknown codebases—anecdotal evidence of cross-tenant leakage before formal triage. Similar issues have been seen in public tools like ChatGPT when users paste proprietary or regulated data.[3][4]
Data at risk in coding assistants
By design, coding assistants see:
- Source code and proprietary algorithms
- API keys and secrets in
.envand configs - Regulatory or audit docs (Markdown, specs)
- PII from logs or debugging examples[3][4]
LLM leakage surfaces not only in outputs but also:
- Logs and caches
- Embeddings and analytics stores
- Future training data
This complicates incident response, compliance, and data lifecycle governance.[3][4]
Tenant isolation as an explicit requirement
LLM/agent security guidance stresses mapping data flows—from prompts to embeddings, tools, plugins, and caches—and placing controls at each exposure point.[2] In multi-tenant platforms, isolation must cover:
- Datasets and training jobs
- Model artifacts and registries
- Inference services and KV-caches
- Agent memory and conversation stores[4]
Without a threat model covering KV-cache and prompt leakage, teams rarely deploy:
- Per-tenant KV namespaces
- KV obfuscation
- Side-channel monitoring/detection[2][8]
💼 Mini-conclusion
Lovable Vibe is best understood as a multi-tenant, cache-sharing LLM service. That framing clarifies who (tenants, projects, sessions) and what (code, secrets, PII, logs) must be protected from KV-cache side channels, model inversion, and privacy leaks.
How to Architect Tenant Isolation: KV-Cache, Scheduling, and Data Paths
Fixing Lovable Vibe–style issues means treating KV-cache and scheduling as security-critical, not just performance features.
1. Per-tenant KV namespaces
Scope every KV operation by a composite key:
KVKey = hash(tenant_id, project_id, session_id, prefix_tokens)
Nothing should be shared across mutually untrusted tenants.[6][7][8] For scale, you can scope by “security domain” (e.g., per-VPC) but never globally across customers.
⚡ Implementation sketch (pseudocode)
def kv_lookup(tenant, project, session, prefix):
ns_key = f"{tenant}:{project}:{session}"
return kv_store.get(ns_key, prefix_hash(prefix))
def kv_insert(tenant, project, session, prefix, kv_state):
ns_key = f"{tenant}:{project}:{session}"
kv_store.set(ns_key, prefix_hash(prefix), kv_state)
2. Global vs per-tenant cache trade-offs
-
Global cache
- Pros: maximal reuse, throughput
- Cons: broad side-channel and data-mixing surface
-
Per-tenant / per-domain cache
- Pros: bounded blast radius
- Cons: higher GPU memory use, more fragmentation, tighter eviction policies[7][8]
For high-security tenants, per-tenant caches are mandatory. Lower-sensitivity workloads may tolerate shared caches within a single security domain.
3. KV obfuscation (KV-Cloak)
KV-Cloak-style methods obfuscate KV states with lightweight, reversible matrix transformations before storage, reversing them only inside trusted contexts.[8]
📊 KV-Cloak-style results
Research shows these can:
- Reduce reconstruction quality to near-random noise
- Preserve model accuracy
- Impose minimal performance overhead[8]
4. Integrate into an MLSecOps architecture
KV controls must live within a broader MLSecOps framework where:
- Ingestion, training, and artifact storage share security policies
- Inference, KV-caches, vector DBs, and agent memories are first-class security assets
- RBAC, audit logging, and config management apply uniformly[4]
5. Keep sensitive data out of prompts
No isolation is perfect. Evidence shows employees regularly paste regulated data into LLM tools, risking penalties such as GDPR fines.[3] Combine:
- User education and UI warnings
- Client-side checks for obvious secrets
- Server-side validation and rejection of high-risk patterns[3]
6. Prompt filtering and redaction
Prompt filtering (e.g., PII detection) and output redaction complement KV isolation so that—even if protections fail—exposed data is less likely to include raw secrets or identifiers.[2][3] This supports GDPR and broader AI compliance.
7. Treat serving and caching as critical infra
Handle LLM serving and caching like databases/queues:
- Strong authz/authn
- Change-managed configs
- Centralized logging and tamper-evident audit trails[4]
💡 Mini-conclusion
Tenant isolation means scoping everything—KV-caches, queues, embeddings, logs—by security domain, then layering obfuscation, filtering, and infra controls to defend against data leaks and misuse.
Red-Teaming and Continuous Testing: Catching Leaks Before Users Do
Even careful designs miss edge cases. Automated red-teaming validates your isolation assumptions under adversarial pressure.
Automated LLM red-teaming
Tools like DeepTeam automate LLM red-teaming for >40 vulnerability types (prompt injection, jailbreaks, PII leaks, bias, history leakage) using multiple attack techniques.[1] They:
- Run locally
- Use LLMs to generate attacks and evaluate responses
- Emit JSON reports that plug into CI/CD for continuous assurance and GDPR-style “72-hour rule” evidence.[1]
⚡ Minimal DeepTeam harness (conceptual)[1]
from deepteam import Audit
def llm_callback(prompt):
return my_llm_client.chat(prompt)
audit = Audit(callback=llm_callback, checks=["pii_leak", "history_leak"])
audit.run_report("report.json")
As orgs move from internal prototypes to public generative AI and agentic workflows, automated security testing becomes mandatory.[5]
Lifecycle security guidance
LLM and agent security guidance emphasizes:[2][4]
- Mapping attack surfaces (prompts, logs, caches, plugins, tools)
- Adding guardrails (filters, policies, constrained tools)
- Monitoring interactions at runtime
- Defining incident response for LLM-specific behavior and data flows
A red-team playbook for multi-tenant KV-cache
To catch Lovable Vibe–style bugs:
- Simulate PromptPeek-like cross-tenant attacks against your serving stack[6][7][8]
- Test for history leakage between sessions (unexpected context carryover)
- Run latency-based probes (TTFT differentials, ordering) to infer cache hits
- Vary tenant/project/session identifiers to verify namespace isolation
KV-cache privacy and PromptPeek research provide concrete techniques to adapt for internal red-teaming.[6][7][8][9]
📊 Why this must be continuous
Model behavior and attack methods evolve quickly. Red-team tools must track new jailbreaks, injections, and side channels.[1][8][9] Treat red-teaming as:
- A recurring CI/CD step
- An input to backlogs, threat models, and user-facing docs
💼 Mini-conclusion
Regular, automated red-teaming focused on KV-cache and prompt leakage could have caught Lovable Vibe’s 48-day exposure in staging rather than after user reports.
Incident Response, Communication, and Long-Term Governance
When tenant isolation fails, technical fixes matter—but so do response and governance.
Immediate triage for prompt leakage
On detecting LLM prompt leakage:
- Freeze or re-scope shared KV-caches to strict per-tenant boundaries
- Disable implicated optimizations (e.g., LPM)
- Rotate secrets/credentials that may have appeared in prompts or code
- Snapshot logs/metrics for forensics while limiting new exposure[3][4]
Notification and transparency
If PII or regulated data leaked, you may face breach-style notification duties under GDPR and similar regimes.[2][3]
⚠️ Communication principles
- Be precise about timeframe (e.g., 48 days), affected components (KV-cache), and data types at risk
- Share concrete remediation plans and timelines
- Avoid vague language that implies poor architectural understanding
Structured root-cause analysis
RCA must span ML and traditional infra:
- KV-cache design and scheduling (namespacing, reuse rules)
- Serving framework configs
- Observability/logging exposure
- Access controls and ML deployment practices[4][8]
LLM issues like KV sharing or agent behavior typically intersect with logging, identity, and CI/CD; they are not “just model bugs.”[4][8]
Governance and risk registers
Mature AI governance should list:
- LLM data leakage and privacy leaks
- Prompt injection and jailbreaking
- KV-cache/embedding side channels
- Data poisoning and model drift
Each item needs:
- Clear ownership across security, ML, and product
- Documented mitigations and escalation paths[2][3]
As AI becomes more autonomous and mission-critical, the cost of trust failures like cross-tenant leaks grows, making proactive governance a differentiator.[5]
Rebuilding trust after a Lovable Vibe–style incident
Platforms in Lovable Vibe’s position should:
- Publish detailed technical postmortems
- Share security hardening roadmaps (per-tenant caches, KV-Cloak-style defenses, robust red-teaming)
- Commission third-party audits focused on KV-cache leakage, prompt isolation, and data privacy controls[6][7][9]
💡 Mini-conclusion
Zero incidents cannot be guaranteed, but you can show you’ve applied KV-cache research, rebuilt with layered defenses, and established governance so future failures are smaller, shorter, and better contained.
Conclusion: Turn KV-Cache Prompt Leakage into a Bounded Engineering Problem
The Lovable Vibe incident shows how a single design choice—sharing KV-cache across tenants for efficiency—can quietly break isolation and trigger a platform-wide trust crisis.[6][8]
By:
- Understanding KV-cache side channels
- Modeling LLM-specific threats
- Treating serving infrastructure as part of the security perimeter
engineering teams can shrink failure blast radius via:
- Per-tenant or per-domain KV namespaces
- Obfuscation mechanisms like KV-Cloak where needed
- Prompt/output filtering plus strict logging and access controls
- Automated red-teaming in CI/CD to catch leaks before users do[1][2][4][8]
If you run a multi-tenant LLM platform, start by mapping where KV-cache, prompts, and logs cross tenant or security-domain boundaries. Then build a minimal red-team harness to probe for KV-cache leakage and prompt bleeding—before attackers or customers find it in production.[1][6][7]
🚨 Absolute length discipline
Designing for isolation, testing for leaks, and treating caching as critical infrastructure transforms “mysterious” AI failures into bounded engineering problems that IT/DevOps, data science, and ML teams can systematically detect, mitigate, and govern over time.
About CoreProse: Research-first AI content generation with verified citations. Zero hallucinations.
Top comments (0)