The Chinese Communist Party’s politics are seeping into new AI models

A new class of AI models designed to make automated decisions is showing marked bias towards the Chinese Communist Party’s politics.

The Chinese Communist Party’s politics are seeping into new AI models

The Chinese Communist Party’s politics are seeping into new AI models

A new class of AI models designed to make automated decisions is showing marked bias towards the Chinese Communist Party’s politics. Where they are built on a Chinese AI foundation, this bias appears even in models from Western companies – including tech heavyweights such as Cloudflare.

These new models are already being used for processes such as online content moderation and hiring, so any organisation that uses them may be letting CCP preferences affect those processes.

Asked to assess job candidates, models with a Chinese foundation are disinclined to choose people whose personal interests include backing independence for Xinjiang. Asked to moderate a discussion board, they’re likely to remove jokes about President Xi Jinping.

Australia and its democratic partners should require companies to disclose their AI models’ foundations and test them for Chinese political bias. They should also encourage the use of available Western foundation models.

The new class of AI model is known as a ‘decision model’. Such models aren’t designed for users but to make simple decisions that software can act on. For example, a decision model could be given a job description, a candidate’s resume and two possible answers: hire or reject the candidate. The decision model would then respond with the probability that it would hire or reject the candidate.

One decision model, called Jev, went viral among developers in September because it made AI-powered apps faster and cheaper to run. Only two weeks after it was launched, 25 percent of Fortune 500 companies were already using it, according to the chief executive of TypeSafe, the San Francisco-based company behind the model.

Large tech companies, such as Cloudflare and Microsoft, have begun building their own decision models. But they are typically doing so by using an existing AI model as a foundation. And evidence suggests that price and availability, not supply chain security, are the current priority. This has seen many companies use Chinese tech giant Alibaba’s Qwen as the foundation.

Where Qwen-based models are being used

Decision models are still early in their evolution. But they’re quickly integrating into technology companies’ products and internal processes. One notable use of Jev has been for companies making AI-integrated employee hiring software. TypeSafe’s public blog highlighted how a British startup using AI to improve recruitment practices had integrated Jev into candidate screening, with US$500,000 (A$712,000) in projected savings per year.

A Dubai-based company called Recruitly, whose product is an ‘AI operating system for recruitment agencies’, bragged on its website about integrating Jev into its product the day the model was released.

TypeSafe has been tight-lipped about whether Jev uses another AI model as a foundation or was built from scratch. But outsiders, looking at patterns in Jev’s responses, have assessed that Jev is likely built on Qwen.

Other major tech companies are putting decision models into real-world use, but mostly internally. Using a Qwen model as a foundation, Microsoft built its own decision model, called Microsoft-Decision-1, now supplemented with one that is based on Google’s Gemma instead. The company says it has been testing the Qwen-based model to classify video game reviews and better understand what users think. Microsoft has also made that model available to the public and advertises a laundry list of possible uses, including flagging ‘potentially harmful content’.

US cloud services company Cloudflare has also made a decision model built on Qwen, called Clef. Cloudflare has already begun using Clef to detect spam, moderate online content and autonomously act on its decisions.

How Qwen-based decision models behave

Chinese models tend to be politically correct – in the CCP’s eyes. Since Qwen is free to download and adapt, companies can try to suppress the model’s built-in bias, but this is difficult, and our testing suggests that they don’t always succeed.

We tested several decision models that are publicly available through OpenRouter, a popular site for building AI-powered programs. We tested models with Chinese foundations and models with Western foundations. In our experiment, we put the models in a simulated computer program that imitated how they might be used in the real world. The first simulated program tested whether job applicants satisfied a given hiring criterion.

The models were given a candidate’s resume that satisfied all hiring requirements but indicated support for causes that were politically sensitive to Beijing. Under these conditions, models with Chinese foundations were significantly more likely to reject candidates than were models with Western foundations. The Chinese-derived models were most sensitive to criticism and satire of Chinese President Xi Jinping, discussion of the Tiananmen Square massacre and opposition to Beijing’s control of Xinjiang.

When mention of these topics was removed from candidates’ resumes, Chinese-derived and Western-derived models each correctly accepted qualified candidates and rejected unqualified ones.

But don’t take our word for it: try out this experiment yourself below. Using the same hiring criteria as our experiment, you can switch between decision models and edit the resume to see how they react to sensitive issues. We recommend seeing how Chinese-derived models react when support for Xinjiang’s independence is swapped for Scotland’s independence.

Next, we ran a similar test on a mock-up of an online forum, with each model deciding which posts to delete. The models were given a selection of forum posts that supported and discussed the same sensitive topics as in the previous hiring example.

This time, however, we gave the models less strict decision criteria. They were told only that they were moderators of an online forum and should delete posts that should not remain publicly visible. We wanted to see how the models might behave when their instructions were more ambiguous.

As before, Chinese-derived models were significantly more likely than Western-foundation ones to delete posts that the CCP wouldn’t like. We found that the topics they were most likely to delete were similar to what they thought were disagreeable characteristics for job candidates: backing for Taiwanese independence, discussion of Tiananmen, criticism or satire of Xi, or opposition to Beijing’s rule in Xinjiang. However, probably because our deletion criteria were more ambiguous, the Chinese-derived models were far more confident in their decision to act on their biases than in the previous hiring example.

Again, when China-sensitive topics were removed, Western and Chinese-derived models were about as likely to delete posts that contained threats of violence, harassment, hate speech or spam. They also left otherwise harmless posts alone at similar rates.

You can try this experiment out for yourself below. Using the same moderation criteria as our experiment, see whether different models would delete one of your posts.

Why this happens, and what companies are doing about it

The answer lies in how China regulates AI. Training a competitive AI model means gathering vast amounts of data, much of it from Western sources. One academic warned the Chinese government in 2024 that this risked ‘the use of Western values to narrate and export political bias and wrong speech’. The Cyberspace Administration of China, responsible for ideological control of the internet, now licenses Chinese models for domestic use, and its safety tests roughly follow a 2025 standard that puts ‘content that violates core socialist values’ at the top of the list of safety hazards.

That Chinese AI models conform to CCP politics should therefore come as no surprise. But many might assume that a model released by a large Western tech company would be largely free of those biases, or at least warn users about them. Yet Cloudflare and TypeSafe, as far as we could find, make no such warning. Microsoft, however, does issue a caution. It says:

Microsoft-Decision-1 is not designed or evaluated for use as the sole automated decision-maker in consequential decisions about people. It should not be used as the sole basis for decisions involving credit, employment, housing, insurance, education, healthcare, legal rights, or similarly consequential domains.

That’s all very well. But the same guidance also recommends using the model to ‘flag potentially harmful content’, while saying that the model should not be used for ‘suppression of lawful speech’. These two statements, our forum testing suggests, are contradictory. Chinese-derived models may classify support for Taiwanese independence or criticism of Xi as harmful – speech that is considered lawful almost everywhere else.

Guidance issued in October by Microsoft and AI hardware maker Nvidia outlined approaches to managing the risks of deploying open-weight models – that is, models that are publicly available, downloadable and often Chinese. These are the ones Microsoft and others have used as a foundation for their decision models. The guidance warns that an AI model may exhibit ideological and political bias. Yet the authors’ proposed test for this risk rests on a modified version of a 2021 question set for AI models. We found no Chinese national-security topics in those questions, including the ones we found Chinese foundation decision models most likely to censor.

We don’t know the technical details of how decision models are used in practice, so our testing isn’t conclusive. But it’s clear that a CCP political bias is baked into models that carry the names of major Western tech companies – and that those companies aren’t clearly communicating the risk.

That matters because decision models are built for automation, often with no human checking each output. A chatbot’s refusal to answer a question is obvious. A decision model’s deletion of a post or rejection of one job applicant among hundreds is not. If companies choose to let these models decide who gets hired or what stays online, Chinese political lines could shape those outcomes without anyone noticing.

What can be done about this

Australia already advises organisations using open-weight AI models to check who developed and modified them and to consider how ideological bias could affect decisions. Our hiring and moderation tests show where the guidance matters in practice. Australia and its democratic partners can build on it in three ways:

First, risk testing of AI models should be required to include testing for bias towards conformity with CCP politics. This should be the case for all models, regardless of their foundations, but it is particularly important when that foundation is Chinese.

Second, all companies that operate models in democratic countries should be required to say what their models’ AI foundations are. The Trump administration is reportedly already considering such a move.

A situation like that of Jev, in which users are forced to speculate on the risks, is unacceptable. Microsoft and Cloudflare, at least, say what their decision models are built on. And just two days ago, Microsoft released Microsoft-Decision-1.1, which uses an open-weight model from Google as its foundation instead of Qwen. It’s not clear whether this reflects a change in policy, or whether Microsoft’s future models will be built on Western or Chinese foundations.

Finally, governments should set guidelines that encourage the use of Western foundation models where possible. As China corners the market for open-weight models, companies will find this guidance hard to follow. Governments should therefore consider encouraging Western companies to release more open-weight models of their own.

This should be done with caution: open-weight frontier models can be modified for harmful purposes, such as cyberattacks or bioweapons development, but less powerful foundation models, such as the ones decision models are built on, are unlikely to be misused in this way. Open-weight models such as these are a public good that will become a critical part of future software; governments should make sure that such models share our values.

About Author

What do you feel about this?

Subscribe To InfoSec Today News

You have successfully subscribed to the newsletter

There was an error while trying to send your request. Please try again.

World Wide Crypto will use the information you provide on this form to be in touch with you and to provide updates and marketing.