Over the weekend, Anthropic chief executive Dario Amodei called for artificial intelligence (AI) companies, including his own, to slow down their work. Sam Altman and Elon Musk, heads of rivals OpenAI and xAI respectively, agreed.
The move reflects concern across the AI industry and, more broadly, about the dangers of new, rapidly improving systems. Recent high-profile incidents such as OpenAI’s AI agents hacking another company and hijacking a public website have shown current systems can break out of safety confines – and even more capable systems are in development.
Further complicating AI safety is the tension between safety and performance. Companies will be reluctant to limit the performance of their models in the name of safety for fear of losing ground to competitors. US President Donald Trump has also rejected calls for a slowdown, for fear of losing ground to China.
This means any successful effort at “pacing the rate of capabilities advancement so that risk prevention has time to keep up”, as Amodei puts it, will require significant cooperation between rival companies – and nations.
Risk minimization
New technologies often bring new risks, and new concerns. Often governments, researchers and companies do find ways to manage those risks.
In 1975, the Asilomar conference on then-new DNA technologies did much to ensure research didn’t get ahead of our understanding of safety and risk. Similarly, in the 1990s, the US government attempted to build a consensus on limiting cryptography and computer security.
The AI situation has an extra twist: the industry is in the middle of a gold rush. Scientific rivals may be able to restrain themselves, but commercial rivals rarely hold back.
There are clear precedents for self-regulation failing in the face of competitive pressure. In the Boeing 737 Max disaster in 2018, for example, pressure to catch up to rival Airbus led Boeing to hide the limitations of the Max, which ultimately cost lives.
In the AI race, the scale of the competitive tension is even greater.
Anthropic and OpenAI are competing to establish market dominance before pursuing initial public offerings (IPOs) that could raise tens or even hundreds of billions of dollars.
And at the nation-state level, the stakes are higher again, with the US and China each hoping to use the new technology for geopolitical advantage.
Politics of a pause
Amodei’s slowdown proposal centers around a three-step plan: embedding independent third-party safety reviewers, establishing coordinated industry safety standards within democratic nations, and eventually securing global agreements.
This would include strict limits on AI chip exports to companies and countries that do not agree to prioritize AI safety.
Anthropic and OpenAI have already agreed to the first phase of this plan, despite their history of suing, publicly insulting, and undercutting each other in pursuit of market dominance.
While recent high-profile hacks may have forced their hands, the leading AI companies may benefit from a development pause or slowdown. For one thing, it could put off strict legislation such as US Senator Bernie Sanders’ proposed Ban Artificial Superintelligence Act.
For another, by creating expensive safety standards and limiting chip exports, it could block smaller competitors – especially Chinese companies such as DeepSeek and Alibaba – from catching up. A pause would also give the overstretched frontier labs a chance to recoup, recover, and focus on profits over progress.
These AI labs have spent enormous amounts to produce their existing models, which must be repaid. At the same time, progress is hitting speed bumps as new high-quality training data gets harder to find, and building the massive data center infrastructure needed to sustain development is a challenge in itself.
A coordinated safety pause could be a convenient public reason for a plateau in AI model performance.
No safe solution
Making AI safe won’t be easy. The governance of software is notoriously difficult, as past attempts to legislate encryption software have shown.
However, unlike other software, AI is dependent on relatively scarce physical hardware. It needs advanced silicon chips and the massive data centres required to power them.
This is where governments have real leverage. Here they have ways to monitor and control AI development, if they can find the legislative will.
In the meantime, there are steps businesses and governments can take to minimise how exposed we all are to AI-driven harms. This should include ensuring that critical safety infrastructure – like power grids, water supplies, and military systems – are not only isolated from AI, but potentially even isolated entirely from the internet.
And the question of liability is also important. It can’t just be the users of AI systems who face legal jeopardy for acts assisted by AI, but also the people responsible for making AI systems.
Fundamentally, harm produced by AI – even by “autonomous” AI systems – isn’t an abstract technological byproduct. It is the direct result of decisions made by both those making AI and those using it.
Andrew Cullen is senior research fellow at the School of Computing and Information Systems, The University of Melbourne.
When there’s a will, there is a way. Just look at what Huawei did.
The Once lauded Free press in US and perhaps elsewhere hinders too much while abetting too little, particularly the socio-polity Middle. It sucks…
The great thing about our friends in the West and their stupidity – sanctions, tariffs and Zionist war: Inflation is up, purchasing power is down and companies are VERY cost sensitive.
China wins on price and quality. It dominates market share. You build something, China can offer 5-10 times cheaper with same quality.
Now tell me, in a world of inflation, is it better to import deflation or more inflation? USA exports inflation. China exports deflation. Which one is better for your wallet? Which one is not weaponized by Club Epstein?
You don’t make any sense to our miserable western amigos. It’s useless. But that’s a good thing. Giving up Nord steam because the U.S. said so was the most stupidiest thing any human can do. It means our German and European amigos are absolute dummies. The average IQ of our western amigos have dropped lower than Africa and the Middle East. I don’t expect them to understand any of the above.
The Western amigos are running on a 500 year old colonial model that has run out of road. And they are trapped in the world’s most insidious propaganda system. It takes real effort to break out of this system. Maybe 5% of the Western amigos understand it. 95% are at a disadvantage since they will be the last to find out the truth about their system. They know something is wrong, but they think the problem is everywhere else but with them. That is what exceptionalism does. It is very embarasing to admit you were fed lies your entire life and you were a dummy who believed colonial propaganda.
My first impression of the West was how much they ALWAYS needed a public enemy to blame. Communism, Islam, Russian, Iran, China. I thought to myself, how funny. A system that claims to be better than the world, yet they are so insecure about themselves they need to always blame someone that coincidentally always aligned with the CIA narrative.
This system is laughable.
How did Mo get to heaven on Burak? Talk about believing fairy tales
The baka Capon is down the rabbit hole in La La Land again.
US is always equal opportunity hater, … States, Regions, People, Religions/Isms, everyone and everything in general… Laughable?
Meanwhile Europe can feed itself, can the little Chinese ?
USA is laughing all the way to the bank
??? Laughing ??? Seems at least 1/4 short, that, Boyo…