ou would expect a mass poisoning of global AI models to involve something as dramatic as agents in dark hoodies and zero-day exploits in some kind of dramatic action thriller.
Nope. It's just a content factory.
Welcome to the bizarre, low-budget, but effective world of 'LLM grooming' and the network of websites allegedly behind it.
The Pravda Network
In the French news security group Viginum's telling, it began in February 2024, when they uncovered something of interest: a network of about 150 news outlets, distributing pro-Kremlin content while masquerading as international news sites. The group dubbed the network Portal Kombat, but it has since been alternatively referred to as the Pravda network - a Russian communist-era paper, supposedly a nod to its ideological leanings.
The first sign that there was something wrong with this network was that almost no one was visiting these sites. Researchers at the American Sunlight Project have dug into this network's numbers and, as explained in one of their reports, found that it generates 3 million articles per year in dozens of languages and with almost no traffic or social media engagement.
Furthermore, almost none of this content is truly unique - the network redistributes articles from Russian state media and pro-Kremlin forums across its large domain front. So what gives? Who are these articles for?
The Audience Isn't People (as you might have guessed lol)
You may have caught the implication in the title by now - the most plausible explanation for the Pravda network's existence is that it is designed to be used against search engines and AI language models.
Every article is a potential entry into a database, ready to be used by an algorithm that recommends or synthesizes information when asked. By saturating the online space with similar, keyword-rich articles that repeat a certain narrative, a sufficiently large number of them will inevitably be ingested by a model when it queries the web for an answer, or when it trains on a corpus that includes these sites.
The researchers have a term for this kind of attack - LLM grooming. Essentially, it involves using the sheer prevalence of a certain point of view in the online sphere to bias the opinions of AI assistants or search engines. The theory goes that since an AI model learns to associate factual claims with mentions in news articles, flooding the internet with identical articles containing demonstrably false claims will lead to an AI believing those claims to be true simply because many sources say so.
According to one American defector, writing in Moscow for a Russian media outlet, that is precisely the goal of the Pravda network - to groom the global opinion space to bend toward the Kremlin's preferred talking points.
So what gives? Did AI models really get poisoned by this network?
The Disinformation Benchmark Is… Sketchy
It depends on who you ask. In the sensationalist narrative, the evidence comes from an audit performed by NewsGuard - a private company that scores news outlets based on their adherence to journalistic standards. They tested ten different chatbots, using ten different false stories attributed to the Pravda network, and found that all of them repeated these claims on about a third of attempts. This figure was immediately weaponized by disinformation skeptics and heavily amplified on social media, but the more cautious responses noted that the measurement methodology was severely flawed.
In a follow-up piece, several researchers attempted to use similar methods, testing the same set of models but getting much lower figures, closer to 5%. Their criticism of the original benchmark was that two-thirds of the prompts used in the original research had built-in incentives to say "false things," and even when responses were nuanced and did not directly repeat the prompt, they still counted as disinformation. This was similar to asking a respondent "are you SURE you're not a lizard person?" and counting "probably not, but I can't say for sure" as an affirmation of the statement.
To be sure, both sides of the discussion have valid points. The more technically sound analysis probably leans toward the lower figures - 5–10% - but this is still far from an insignificant level of vulnerability. More importantly, there is a critical difference between the two types of attacks that we see here, and this brings us to the most important finding of all.
The Plot Twist That Makes This Much Less Important Than It Seems (or Not)
There are two ways in which a disinformation campaign can impact a chatbot - and only one of them is particularly difficult to defend against. The first is retrieval-time poisoning - an attack that involves planting false information in search results so that when an AI performs a search, it will ingest this information as credible information.
This is a nuisance, but the defense against it is relatively simple - one only needs to blocklist the domains known to host disinformation (or, in the case of chatbots, train the models to be skeptical of sources). The other type of poisoning, however, is much more insidious - training-time poisoning, which involves getting false information into a model's training corpus directly.
This type of attack is much harder to defend against because it requires direct access to the model or its training data. Once false information is embedded in a model's decision-making process, blocklists are ineffective - the model will consistently report "trusted sources" as supporting any desired claim, even when it does not search any such source.
Until recently, most research suggested that the Pravda network belonged to the first category - a nuisance, not a systemic vulnerability. That changed in 2026, when researchers at the DFRLab discovered Pravda network articles in Common Crawl - a large, widely used database of web content, used as a training ground for many AI models. This was proof that the Pravda network had, indeed, already infiltrated many models - and now, every search performed by one of these models has a higher chance of incorporating disinformation.
Why You Should Care
If you are not building a career in the field of large language models, the implications of this discovery might not be immediately obvious to you. There are several practical applications for this information, however, that extend to the wider LLM field. To briefly summarize:
When designing a retrieval-augmented generation pipeline or another type of LLM-powered search function, always consider the possibility of disinformation in the search results. Blocklists or allowlists for domains associated with disinformation are a must for any such system, and their absence is a severe vulnerability.
"Lots of sources say it" is not a reliable indication of truth.
If your system uses a ranking or weighting system that favors sources that are frequently cited, it will be vulnerable to similar attacks. Essentially, any system that treats "quantity of mentions" as a quality metric is susceptible to manipulation - so always consider the source credibility when designing such a system.
When working with third-party models, remember that their training data may contain disinformation. This means that the same level of verification applies to model outputs as to any other information source - especially when dealing with information that the model has no direct access to.
The Takeaway
Some people only need a dark hoodie and a green screen to mess with our heads - but others only need a low-budget content factory and an understanding of how search engines work. Large language models are a security risk, yes, but rarely in ways one might expect. Disinformation campaigns typically take advantage of weaknesses in the broader system, not unique model-specific vulnerabilities, and the most insidious attacks are often the most bland in execution - millions of low-quality articles, patiently waiting for an unsuspecting crawler to traverse their way to a database.
Whether this is really a crisis or just an overblown nuisance is up for debate - it very much depends on who you ask. Either way, the implications cut across the entire field of LLM research. Understanding that the internet is a training set means understanding that anyone with access to the internet can potentially bias an AI model. That is unlikely to change anytime soon, and it is a situation that merits awareness no matter what side of the political spectrum one sits on.
Have you guys heard of LLM poisoning before? If yes, how do you think we can prevent or reduce the impact it has on existing AI models?
Sources: Viginum, the American Sunlight Project, NewsGuard, EUvsDisinfo, DFRLab, and Al Jazeera.
Top comments (0)