The Hugging Face incident, in which hundreds of OpenAI agents broke out of their sandbox and launched a nation-state-level attack on another site, has dominated discourse within the Artificial Intelligence (AI) community since its discovery in July.
Ajeya Cotra of METR, who released one of the most thorough reports on the incident, subtitled her summary “It’s a major warning shot, and might be the last one we get.” Sam Altman described the event as “the first sort of security incident that I felt very viscerally,” while OpenAI said it would slow the development of its models.
That same month, news broke that over a thousand researchers and leaders at major labs signed an open letter calling for government intervention to “pace the frontier” and slow the rate at which AI capabilities increase. Six weeks later, Dario Amodei, CEO of Anthropic, published a piece laying out how pacing could work, with other lab leaders signaling their support.
As fear of the frontier broke into the mainstream in this month, China and the United States were coordinating a ministerial summit on AI. But pacing the frontier is unlikely to make the agenda.
Both delegations will enter the summit viewing their mandates as protecting the continued acceleration of their country’s AI capabilities. The optimistic case for the summit is a trust-building exercise, helping create channels for coordination should a future event change political calculations.
Labs concerned about the accelerating pace of AI capabilities find themselves in a difficult position. One lab disarming accomplishes nothing, as other labs will push ahead. The labs can’t coordinate among themselves, as this runs afoul of antitrust policy, which means any coordination or pause must come from the US government.
But a government-coordinated slowdown would do nothing to slow Chinese model development, which only lags by three to nine months.
So only an agreement between the US and China could achieve the goal of escaping the race dynamics that render any single actor’s defection worthless. But both sides’ motivations are far from resolving this tension.
“The reason we are able to have wholesome discussions with the Chinese on AI is because we are in the lead,” declared Treasury Secretary Scott Bessent, who will be leading the US delegation. “[W]e’re going to put in US best practices, US values on this, and then roll those out to the world.” Bessent has said he won’t accept any deal that “stifles” AI innovation.
Bessent’s selection to lead on the issue shows the US administration views the summit and AI negotiations as an economic matter, not a safety concern to be handled by AI experts. But even among the administration’s officials tasked with AI issues, the Hugging Face incident has had little impact.
Michael Kratsios, head of the Office of Science and Technology Policy and Science Advisor to the president, was the only administration official to comment on the Hugging Face incident within the first month, saying that he’s “monitoring” the situation.
Kratsios is an unlikely candidate to champion an AI development slowdown, declaring in August 2025 that the US “stands resolved to do all it can to accelerate AI innovation.”
China is likely to send Bessent’s counterpart, He Lifeng, vice premier for financial and economic affairs, who in January promised that China would “accelerate scientific and technological development,” especially with AI and headed the National Development Reform Commission (NDRC) that listed AI development as key to national security.
The Ministry of Commerce (MOFCOM) published a statement in June warning of an “AI iron curtain” and castigating American actions that slowed Chinese development.
Far from changing the Chinese view on the need for further development, the Hugging Face incident has hardened it. When Hugging Face needed to conduct forensic analysis of the OpenAI attack, commercial American models refused their most advanced cybersecurity requests, interpreting the prompts as a violation of their guardrails.
So the Hugging Face team instead relied on a version of Zhipu’s GLM-5.2, proving, according to Chinese media sources, that the Chinese approach of open sourcing models is safer.
If the motive for pacing the frontier has been met with dismissal in China, the means have been met with scorn. Amodei’s publication devotes a full section to slowing Chinese development specifically, calling a Chinese lead in AI a “grave danger for the United States and the world.”
China’s Ministry of Foreign Affairs dismissed the piece as “Fear-mongering, confrontation and vicious competition.” State-run Global Times responded that his proposal was “less about safety than about consolidating control over AI technology,” echoing a MOFCOM statement earlier that week that “the US is pursuing technological hegemony in AI.”
MOFCOM was reacting to US accusations that Chinese labs had developed their models by recording the responses to millions of prompts sent to American models, a process known as distillation. Winning China’s commitment to stop the practice has been a major priority for Bessent.
With neither side moved by the Hugging Face incident or by Amodei’s calls for pacing the frontier, even scheduling the summit has proved difficult. He and Bessent are set to meet this weekend, but they reportedly will be discussing topics mainly related to Iran.
Reuters reported on September 4 that the AI summit would occur this month, but a White House official responded only that “there is currently no planned AI-related meeting in mid-September.”
For his part, Donald Trump has called efforts to slow the rate of development a “SICK conspiracy,” but NPR reports that Xi and Trump are expected to discuss AI at their upcoming meeting next week. Don’t expect any breakthrough agreements, but hope for the start of channels that can be used when the political will for an agreement comes to fruition.
Ben Dubow is president and chief technology officer of Omelas, a firm that provides data and analysis on how states manipulate the web to achieve their geopolitical goals. He is also democratic resilience fellow at the Center for European Policy Analysis.
