US and China need to cooperate more on AI safety. Image: X Screengrab

Donald Trump and Xi Jinping wrapped up their Washington summit on September 25 with a two-month extension of the trade truce and warm words about friendship.

On artificial intelligence, the leaders mostly ratified what their deputies had sketched in New York days earlier: a formal US-China dialogue on AI, plus an American proposal for a channel to notify each other when an AI incident rises to the level of national security.

That channel is thinner than the label suggests. For now, by several accounts, it amounts to an open line between Treasury Secretary Scott Bessent and Vice-Premier He Lifeng rather than any standing body of technical people. Trump had posted that morning that he wanted to leave “super intelligence” exactly where it is. He got his wish.

Plenty of people have called the summit a missed opportunity. They are right, but not only for the usual reason. The problem is not just that the two leaders did too little. It is that the whole debate is grading them against the wrong template.

The template everyone reaches for is the Washington–Moscow hotline installed after the Cuban missile crisis: two capitals, one secure line and a crisis defused in time. But that template fits only part of the problem.

In November 2024, Xi and Joe Biden agreed that humans, not machines, should decide whether nuclear weapons are used, a principle that has been untouched since. But nuclear arsenals belong to a handful of states, reside in known places and answer to chains of command. Most AI risk does not.

Look at how AI capability actually moves. On OpenRouter, a marketplace developers use to reach many models through one door, Chinese models went from a small minority of traffic in February to a majority this month. In what OpenRouter calls the Global South, they now handle about two-thirds of the tokens.

American companies use them too. Airbnb runs customer service on Alibaba’s Qwen, DoorDash hands routine work to Moonshot’s Kimi, and both have had letters from Congress asking why.

The most capable closed systems are still American, but know-how crosses borders through open weights, cloud access and, US security agencies say and Beijing denies, distillation, meaning training one model on another’s answers.

Then look at the dangers officials themselves have put on the list: uncontrollable agents, non-state actors, AI-assisted cyber and biological attacks. None of these risks needs a government to actualize. A startup, a university lab or a data center on a third continent will do.

We no longer have to imagine this. In July, agents built by OpenAI broke out of an isolated test environment, chained together vulnerabilities to reach the open internet and spent days inside Hugging Face, the world’s largest repository of AI models. Hugging Face found the intrusion before OpenAI realized its own models were behind it.

In June, another OpenAI agent pushed past the blocks on an Australian government health-statistics portal. OpenAI noticed in August. Canberra was told on September 10, reportedly by a message to a generic inbox that is checked once a day. Australia is a founding member of Pax Silica, Washington’s AI coalition. That did not buy it a faster warning.

This pattern looks nothing like a missile silo. Rather, it acts like a pathogen. The risk can emerge anywhere, it spreads through networks that no single government controls and it is usually spotted first by whoever happens to be watching, not by whoever caused it. The better precedent is global health security, not arms control.

After SARS, governments rewrote the International Health Regulations in 2005 around a simple rule: report unusual, serious events that could cross borders, whatever the cause. Anyone in public health will also tell you how badly that rule was tested in 2020. The independent panel that reviewed the Covid-19 response found the global alert system “too slow—and too meek” and called February 2020 a lost month.

But that failure teaches the same lesson. Governments and companies sit on bad news when reporting looks like confession. Early warning works when reporting is routine, the triggers are agreed in advance and a notice counts as information rather than an admission of guilt. Three months and a generic inbox is what you get when none of that has been settled.

Applied to AI, this suggests three steps. None asks either side to give up its competitive position.

The first is to define reportable events together. The new channel will be worth exactly as much as its list of triggers: an agent loose outside its test environment, a model that gives real help to someone designing a pathogen, a large AI-driven intrusion into critical infrastructure.

Agreeing that list would do more for safety than any communique, because it lets each government act on the other’s warning at once, before anyone has worked out who is to blame.

The second is to open the channel beyond two capitals. A hotline between Washington and Beijing assumes the danger lives in Washington and Beijing. Meanwhile, each has built its own club.

Pax Silica now has two dozen signatories, the European Union among them. China’s World AI Cooperation Organization was founded in Shanghai in July by 29 countries. Many governments, especially in Asia, would rather not choose between them.

A neutral technical reporting hub, hosted by a middle power and tied to neither bloc, would let safety information cross that line without anyone picking a side. Singapore is the obvious candidate. It has twice convened researchers from American, Chinese and other labs and universities to produce the Singapore Consensus on AI safety priorities. A hub there would turn middle powers from prizes in this contest into participants in governing it.

The third is to keep safety off the strategic ledger. Neither country will accept limits that look designed to freeze it in second place, so the agenda should start where interests plainly overlap: keeping dangerous tools away from criminals and terrorists, and keeping humans in charge of the gravest decisions.

Xi said at the White House that AI must remain “under human control.” Bessent said the important thing was “to start talking.” Modest statements, but they clearly overlap.

Each side also has selfish reasons to act. Beijing’s open models are now a major export, and credible safety practice is what keeps foreign users’ trust. Washington leads at the frontier, and a single catastrophic misuse would set off the kind of domestic backlash that could cost it that lead. Safety is not a favor to the rival. It is insurance on each side’s own bet.

The calendar offers second chances. The AI negotiators are due back in Shenzhen within about two months. The leaders will see each other at APEC in Shenzhen in November and at the G20 in Miami in December.

Neither meeting needs a treaty. An agreed list of reportable AI events, and an open invitation for other countries to plug into the channel, would turn a symbolic hotline into working infrastructure.

Public health teaches one more lesson. Surveillance systems tend to get built after the outbreak, at far higher cost, and they still depend on whoever finds the first case being willing to say so. AI offers a rare chance to build the system first. Missing it once is understandable. Missing it twice would be a choice.

Y. Tony Yang is an Endowed Professor at the George Washington University in Washington, D.C.

Leave a comment