Will China Forbid Open-Weight Models?
Yes, if they take Mythos and the US government seriously
Should we expect China to allow open-weight models indefinitely? Or should we expect new industrial policy forcing Chinese labs to keep their weights secret, as capabilities advance to the level of Mythos?
Weights are the parameters encoding all the knowledge and behavior of an LLM. Most frontier models from US labs keep their model weights secret, meaning they are difficult to inspect and modify directly. Though competitor labs, particularly in China, are known to use model distillation in an attempt to extract the capabilities of a proprietary model, distillation is subject to extensive mitigation efforts and has a high potential cost.
Open-weight models are the AI equivalent to open-source software. Rather than keeping the weights secret, some labs publish the entire set of weights for a model, allowing users to run the model however they want, including on their own machine. US labs publish their own open-weight models (Google’s Gemma is a notable recent example), but generally their open-weight models are weaker than their proprietary models. No US lab has published a state-of-the-art (SOTA) model with open weights.
Chinese labs have taken the opposite approach. The vast majority of their frontier models are open-weight. Research with Claude suggests that about 75% of all Chinese models are open-weight, compared with about 40% of US models, but this likely understates the difference when accounting for capabilities. Of the 9 Chinese models included in the below leaderboard from Artificial Analysis, only 1 of them (Qwen3.7-Max) is closed source.
Why have Chinese labs adhered so uniformly to an open-weight approach? I’d guess they have decided this is a comparative advantage against more capable US models. Namely,
Open-weight models allow Chinese labs to target a different business segment than US labs. For example, companies that place a premium on running inference on their own infrastructure, or even locally.
Open-weight models help capture mindshare. Some people are really passionate about open access! And it does undercut the dominance of US labs if a Chinese lab can create a comparable model and make its weights freely available.
Since open-weight models can run on the customer’s own infrastructure, it saves labs compute in the face of chip sanctions (Though these seem counter to the labs’ actual business model? So, I’m a bit skeptical of this one).
Another theory might be that Chinese labs have a stronger culture of open-source software, or that they are less concerned about AI safety than their American counterparts.1 But I imagine the impact from either would be marginal.
No open-weight model has yet topped a standard benchmark, but Chinese labs remain competitive. On June 13th Z.ai released a new open-weight model, GLM 5.2, which has bested all but the most capable proprietary US models. On the above leaderboard, GLM 5.2 is ranked as the most “intelligent” open-weight model, and fourth overall, coming only behind Anthropic and OpenAI’s own SOTA models. Code Arena ranks GLM 5.2 as second only to Fable 5 (currently unavailable!) on web development.
Observers are reasonably skeptical of “benchmaxxed” models, which do well on standard tests but underperform on real-world tasks. This is a frequently cited flaw of Chinese models, which optimize for benchmarks that help garner needed public attention. But GLM 5.2 arrives at a time that many developers and researchers are hungry for open-weight alternatives to proprietary models, spooked by the recent drama with Fable 5.
As I wrote last week, many in the AI community were upset at Anthropic for safeguards which seemed broad and even deceptive. This had already led to arguments on the importance of open-weight models, free from corporate censorship. The US government has since issued an export control directive forcing Anthropic to shut down access to Fable 5 and Mythos. Though it’s convincing evidence against critics who’d thought Anthropic’s approach pusillanimous (even Anthropic’s “excessive” safeguards were not cautious enough!), this has only fueled desire for models which will remain firmly under the user’s control.
Open-weight models are a compelling option for “mid-tier” powers, who are otherwise lost in an AI race dominated by two superpowers. The Trump administration’s export control directive highlighted that even US allies cannot guarantee access to future frontier intelligence. Economic growth and hard power will increasingly depend on AI models. If access to the most capable models could be lost at any time, then a nation exposes itself to massive economic and even military risks.
China is then in the attractive position of producing a type of model which should become increasingly attractive to nations around the world. Though it’s unclear whether a European democracy would be comfortable building critical infrastructure on top of models unable to answer basic questions about Tiananmen Square, the same will not be true of all nations. International companies may also find themselves happy to compromise on politically sensitive topics.
China has a huge opportunity to wield technological soft power among nations wary of an unreliable US ecosystem. But I think it is very unlikely that China will continue to release SOTA models with open weights. Over the next year, we should expect the PRC to grow increasingly concerned about the capabilities of frontier intelligence and implement industrial policy mandating frontier models remain closed weight.
Motivations can be found already in the actions of the Trump administration, which has demonstrated serious concern over the cybersecurity capabilities of Mythos-class models. It is true that the administration’s actions are muddled by an apparent disdain for Anthropic. But the PRC will take note if the US has begun to restrict model access on the basis of security concerns.
Open-weight models are much easier to jailbreak than closed-weight models. Direct access to the model weights means an attacker can change model parameters to update or remove safeguards. An attacker must still determine which weights implement a safeguard, but at least they are not limited to prompt injection. Moreover, an open-weight model can be downloaded. Attackers are restricted by their own resources in running a jailbreak, with no risk of detection. When the attack is complete, the jailbroken model can then be uploaded and shared at will.
Is it more to China’s advantage that foreign and domestic entities have access to powerful models which are potentially jailbroken, or that they should have to work with proprietary models gated by Chinese labs?
Suppose Z.ai were to release “GLM 6.0,” the first Chinese model to match the capabilities of Anthropic’s Mythos. In theory, it should have all the same cyber- and biosecurity risks as Anthropic claims for Mythos/Fable. Unfortunately for Chinese labs, Anthropic has already released Fable 5.5, so they are still several months behind the frontier. Will the PRC allow Z.ai to release GLM 6.0 as an open-weight model, preserving a comparative advantage for its economy, or will they require GLM 6.0 to remain closed-weight, ensuring cyber capabilities are not shared with potential adversaries?
All dynamics seem to point toward the latter. Particularly in a regime where the US has started to prevent foreign powers from accessing its most advanced models, why would the PRC freely surrender its own advantages in machine intelligence? Just as importantly, why would the PRC accept the risk of greater instability from foreign or domestic entities gaining jailbroken access to a theoretical GLM 6.0?
In what world does the PRC ignore a domestic lab producing levels of intelligence which freaked out the US government, then accept it when the lab shares that intelligence freely with the entire world? This is inconsistent with the history of the PRC, or maybe any self-interested nation-state.
Just as US labs do continue to produce open-weight models, I would not expect Chinese labs to suddenly stop. But the recent coverage of Mythos/Fable and the US government’s interventions will not go unnoticed in Beijing. Over the next year, we should expect China to implement industrial policy, openly or secretly, forbidding labs from sharing the weights of their most capable models.
Why is this Relevant for Safety?
In the future, open-weight models may be a key source of AI risk. Because they are easily jailbroken, we have to worry that a future jailbroken “GLM 6.0” could be used by terrorists to produce e.g. bioweapons. Because open-weight models can be downloaded and run on hardware anywhere, a more capable model could be exploited by a rogue foreign lab to conduct automated AI research and kick off a recursive self-improvement loop. For now, these concerns seem theoretical. But maybe a Mythos-class open-weight model is already smart enough that these concerns become realistic!
Increasing secrecy around model development has its own dangers. We lose insight into capability gains on either side of the US-China AI race, and risk a Cold War-style stockpiling of advanced intelligence. But, given the risks associated with open-weight models, it may be unavoidable that governments stop the proliferation of highly capable models. This would anyway be a requirement for any global treaty carefully regulating intelligence advancement.
I have heard a generalization that China is more culturally accepting of technological change and the onset of AI. However, I’m deeply unaware of what AI safety culture in China is like. From a few conversations with colleagues, I’ve gotten the impression it is vastly underdeveloped compared to the US, especially when considering x-risk problems. But I have no hard evidence for this and would love to see more research here.