Two items in a weekly recap packed with crypto heists and nation-state intrusions involve AI tooling, and neither one is about AI.
Look at what actually happened. Parameter Labs disclosed a patched ChatGPT flaw where a malicious shared conversation ran attacker-controlled JavaScript inside the victim's authenticated session, which meant someone else's private conversation content was up for grabs. Separately, a tampered Claude Code extension on the Cursor Marketplace dropped an SSH backdoor on up to 5,000 users.
Read those again. One is stored XSS. The other is a poisoned extension in a marketplace. I have been writing about both of those for a very long time, and neither needed a large language model to exist.
Nothing here is new
The shared-conversation bug is the oldest trick in the web app book: take content from one user, render it in another user's authenticated context, and forget to treat it as hostile. The "AI" part is just that the content happens to be a chat transcript. If you swapped in a shared document or a forum post, the severity story would read the same way. It was patched, which is good, and the interesting question is how many other "share this" features in AI products got the same lazy rendering treatment.
The extension story is the VS Code marketplace problem wearing a new hat. Developers install extensions with broad local access, marketplaces vet them lightly, and an attacker only needs to ride a name people already trust. An SSH backdoor on a dev machine is nasty precisely because of what lives on dev machines: keys, tokens, repo access, cloud credentials. The summary doesn't say how the tampering happened, so I won't guess. But the blast radius of a compromised developer endpoint has never depended on what the extension claimed to do.
The hype check
Here's what I'd push back on. When these stories get framed as "AI security" incidents, it implies a new discipline with new rules, and that framing mostly benefits people selling new categories. The boring truth is that most real-world damage in this space comes from classic appsec failures and classic supply-chain exposure, now attached to products with huge, fast-growing user bases and a lot of trust.
But the opposite take is lazy too. "It's just XSS" undersells why it matters. People paste genuinely sensitive things into chat assistants: internal code, customer data, half-formed strategy, credentials they shouldn't. A chat history is a richer target than most session data used to be. Same bug class, much juicier loot. And AI coding extensions are especially attractive to attackers because the people installing them are, by definition, people with access to valuable things.
So the vulnerability classes are old, but the incentive math has changed. That's the part that's understated.
What this means if you build things
If you ship any feature that lets users share AI-generated or AI-adjacent content, treat the shared artifact as untrusted input. Every time. Sanitize on render, scope the session, assume someone will craft a transcript designed to hurt the person who opens it.
If you manage developers, the extension question deserves an actual answer rather than a shrug. Do you know what's installed across your team? Does anyone review what an extension can reach on the machine? Most orgs I've seen have a policy for npm and nothing at all for the editor itself, which is a strange gap given that the editor sits next to the keys.
And a small note on the larger recap: it's a grab bag of unrelated incidents, from a crypto theft to a DNS compromise to a long-running state-linked actor. The AI items are two lines in a long list. That proportion feels about right to me. The attack surface is growing, but it isn't yet a separate world.
The question I keep circling
If the root causes are this familiar, why do new AI products keep shipping with the old mistakes? Is it speed, is it the assumption that a chat interface is somehow safer than a web form, or is it that the teams building these things are strong on models and thin on appsec? And if it's the last one, who's supposed to be catching it before users do?
— Cor, Skyblue Soft
Sources
AI-assisted draft or imaging, human-curated, reviewed and edited.
Top comments (0)