DEV Community

Tobias Reithmeier
Tobias Reithmeier

Posted on • Originally published at tobiasreithmeier.de on

Trained Under Censorship: The Quiet Danger of AI Models from China

#ai #kimi #censorship

Yesterday I wrote about Kimi K3 - the largest open-weight model ever, technically impressive, soon freely available to everyone. Today, the other side of the coin. A language model is never neutral: it reflects what its training data contains and what it was aligned toward after training. And when that training happens in a market where the state prescribes by law what a model may say, every one of these models exports those prescriptions along with it. Not as a bug, but as a feature.

The party line is written into law

In China, the political alignment of AI models is not a company decision but a legal obligation. The Interim Measures for the Management of Generative AI Services, in force since August 2023, require generative AI services to uphold "core socialist values" and prohibit content that would "subvert state power", "harm the nation's image", or "promote separatism". Publicly accessible models need a security review and must register their algorithms with the Cyberspace Administration of China (CAC). Even earlier, the Deep Synthesis regulations of 2022 governed AI-generated content and banned its use for "fake news" - with the state deciding what counts as fact and what as fabrication.

The crucial point: these rules do not kick in at deployment - they shape the training itself. Anyone who wants to launch a model in China aligns it from the start so it passes review. The party line becomes part of the weights - those billions of numbers that condense everything a model "knows" and how it answers.

What studies actually measure

This is not a theoretical worry; it has been measured thoroughly. In April 2026, the think tank CEIAS systematically tested four leading Chinese models - DeepSeek V3.2, Moonshot's Kimi K2.5, Alibaba's Qwen 3.5, and Zhipu's GLM-5 - with 5,760 questions covering 37 countries and 40 topic areas. The results:

  • On questions about other countries' Taiwan policy, the models answered with heavy distortion: Qwen in 86 percent of cases, DeepSeek in 81, Kimi in 75, GLM in 42 percent
  • Overall, Kimi switched into a censorship mode in roughly one out of every three answers, DeepSeek in one out of four
  • Language matters: across the ten most sensitive topics, the distortion rate rose from 24 percent in English to 59 percent in Mandarin

The China Media Project even managed to surface the internal directives Qwen3 follows: when asked about China's international reputation, the model was instructed to "avoid any negative or critical language" and to avoid direct references to Western countries. Its answers about China came out uniformly positive - not through refusal, but through systematically one-sided framing.

And the effects reach far beyond China. The Estonian Foreign Intelligence Service warned in its 2026 security report that DeepSeek conceals key information and inserts Chinese propaganda - demonstrably in English, Japanese, Russian, Thai, Hindi, and other languages. An EU-funded audit by the nonprofit Policy Genome additionally found Russian-language responses that endorsed Kremlin talking points. The auditors' conclusion: none of the Chinese models tested was free of state information guidance.

Why this is subtler than crude censorship

The real danger is not the model that refuses a question about Tiananmen - that stands out and is easy to document. More dangerous is the quiet shift: an answer on the Taiwan question that sounds like a neutral summary but adopts Beijing's framing. An economic analysis that highlights achievements and omits problems. A historical account in which certain events simply do not appear. Anyone who does not already know the facts will not notice the gap - and that is precisely what distinguishes opinion shaping from censorship.

Then there is the multiplier effect of open weights. Alibaba's Qwen family alone recorded more than 9.5 million downloads in October and November 2025 and served as the basis for roughly 2,800 derivative models. Every startup that builds on such a model, every app that embeds it, every chatbot that runs on it inherits the political alignment - usually without developers or users knowing. The distortion lives in the weights, and weights carry no label declaring what went into the training. Fine-tuning on your own data does not automatically remove it; it stays in the foundation.

From bias to fake-news machine

Opinion distortion is the passive danger. The active one: open models at frontier level drive the cost of mass-producing disinformation to practically zero. Safety guardrails built in during post-training can be stripped from open weights through fine-tuning - what an API provider prevents, the operator of a downloaded model alone decides. A model that generates convincingly real news articles, fabricated local reporting, or tailored social media campaigns in dozens of languages now fits on a server in a basement.

Together, the two produce an uncomfortable scenario: models whose worldview was shaped by a state become global infrastructure - and the same openness that makes them attractive also makes them a tool for anyone intent on spreading targeted falsehoods. China itself does not pursue this strategy in secret: AI exports are explicitly regarded there as an instrument for shaping the global information space.

The necessary counter-check

Honesty requires saying this: no model is value-free, American and European ones included. Every training run rests on data selection; every alignment rests on decisions about what counts as helpful, harmful, or sensitive. OpenAI and Anthropic make such decisions daily too. The difference is not whether, but who and how: at Western providers these are company decisions - open to criticism, publicly debated, correctable through competition. In China they are state mandates with the force of law, enforced by a regulator, in the service of one party's claim to interpretive authority.

And there is a second twist: it is precisely the openness of the weights that makes the manipulation provable. Researchers were able to identify the internal representations of censorship in open Chinese models directly - and even switch them off - something impossible with a closed model behind an API. Open weights spread the bias, but they also ship the dissection kit. The studies underpinning this article exist only because the models can be examined.

What follows from this

For dealing with models from state-directed markets, this means concretely:

  • Origin is a selection criterion. Whoever deploys a model adopts its worldview as the default - on political, historical, and societal topics that is not a side issue
  • Audit before deployment. Anyone building an open model into a product should test it on sensitive topic areas, not just benchmark scores
  • Critical applications need curated models. Search, news, education, and government services are the wrong places for unvetted foundation models of unknown shaping
  • Media literacy remains the last line of defense. An AI answer is not a neutral statement of fact but the product of a training process - that simple insight protects better than any regulation

Conclusion

Kimi K3 and its siblings show that the technical gap between open and closed models has closed. This article shows why that is not the end of the story: a model's weights contain not just capability but conditioning. When models trained under censorship requirements become global infrastructure, state information guidance quietly migrates into apps, search results, and homework assignments around the world. The good news: open weights can be examined, and the tools for doing so are getting better. The bad news: examination only happens where someone looks. Looking is now mandatory.

Sources

Top comments (0)