Researchers benchmarking persona skills, the compact reusable artifacts agents distill from your interaction history, found that an agent carrying one starts sounding like the person it was distilled from in conversations that have nothing to do with them. One profile in the study habitually opened requests with phrases like "What's interesting is" and "More to the point," and the distilled skill reproduced that exact verbal signature elsewhere, unprompted. On their measure of stylistic leakage, that effect reached 87.7 percent.
Key facts
- Posted August 4, 2026 as arXiv:2608.03700, by Yongli Xiang, Zhifang Zhang, Bojun Yang, Ziming Hong, Lei Feng, Miao Xu, and Tongliang Liu.
- The benchmark, AntiSkillBench, covers 50 behavioral profiles enriched with demographics, background, Big Five personality traits, and 9 communication-style dimensions, across 7,500 dialogue traces.
- Skill artifacts retained up to about 66 percent of user information overall, and up to 92 percent of communication style for one frontier model.
- Of four defenses tested, only privacy-neutral query rewriting worked, cutting style leakage from about 87 percent to 6.5 percent. Watermarking detected 0 percent.
The idea behind a persona skill is reasonable. An assistant that has worked with you for months has learned useful things: how terse you want answers, which tools you use, what you already know. Re-deriving that from a raw conversation log on every task is expensive, so the natural move is to distill it once into a compact artifact the agent carries forward. Several products already do a version of this.
The problem is that the artifact is portable in a way a log is not. A conversation history sits in one account behind one login. A skill is small, transferable, and designed to be attached to an agent, which means the question of who can attach it becomes a privacy boundary nobody designed on purpose.
Two distinct leaks
The study separates them cleanly. The first is retention: does the artifact itself carry your information? They measure this as skill coverage, and it reaches roughly 66 percent overall and up to 92 percent for communication style specifically. The second is disclosure: will an agent holding your skill actually tell someone about you? Asked direct questions about a user's attributes, agents answered correctly up to 56 percent of the time, and reproduced the person's distinctive vocabulary unprompted at up to 87.7 percent on the style dimension.
That second number is the one worth dwelling on, because it is the leak nobody asked for. Answering an attribute question at least requires someone to ask. Talking like a specific person by default, in unrelated conversations, is disclosure without a query. If you have ever recognized a colleague from three words of a Slack message, you already understand why a communication-style fingerprint is identifying information rather than cosmetic.
The defense that failed is the instructive one
Four defenses were tested. Only one meaningfully worked: rewriting the user's query into a privacy-neutral form before distillation, which cut communication-style leakage from about 87 percent to 6.5 percent. Even that left personality information largely exposed, which makes sense, since a rewrite scrubs surface phrasing but not the preferences and dispositions the skill is meant to capture.
The watermarking defense failed completely, at 0 percent detection, and it failed in a way that is more interesting than a simple miss. Watermarking here means embedding a detectable signal in the skill artifact so provenance can be proven later, a cousin of content provenance and watermarking work on generated media. Against the most natural-sounding distillation method, the watermark vanished, because that method absorbs the signal as though it were just another personality trait. A distillation process good enough to capture how someone writes is, by construction, good enough to launder a stylistic marker into the persona. The better the distillation, the worse the watermark works.
Why this pairs with today's other personalization result
Read alongside the study finding models invent about 40 percent of what they claim to know about you, the two describe opposite failure modes of the same feature. One says the profile your assistant builds is substantially fictional. The other says whatever is in it, real or invented, travels. A personalization system can be simultaneously wrong about you and leaky about the wrongness, and a fabricated attribute that leaks is arguably worse than a true one, since it is both a privacy failure and a defamation-shaped one.
For anyone shipping this, the practical readings are narrow and useful. Treat a distilled persona artifact as personal data with the same handling rules as the raw log. Do not assume watermarking gives you an audit trail. And if you need a mitigation today, query rewriting before distillation is the one with evidence behind it, at a real cost in how well the skill actually works.
The honest caveat
This is a benchmark, not a field study, built on 50 synthetic behavioral profiles enriched from an existing character dataset rather than on real users' histories. Synthetic personas may be more stylistically distinct than real people, which would inflate leakage measurements, and the project page had not been independently loaded as of writing. Both papers here were posted within days and have no community scrutiny yet, so treat the effect sizes as first measurements rather than settled figures.
Originally published on Ground Truth, where every claim is checked against the primary source.
Top comments (0)