Anthropic chief: AI will ‘test who we are as a species’
The co-founder of the £350bn AI giant Anthropic has warned that “humanity is about to be handed almost unimaginable power” as the technology advances and warned that “it is deeply unclear whether our social, political, and technological systems possess the maturity to wield it.”
In a 20,000 word essay, Dario Amodei, warned that the development of AI will “test who we are as a species” and described his article as “an attempt to jolt people awake.”
Amodei co-founded Anthropic in 2021 and went on to launch Claude, one of the leading AI platforms. His warnings focus on the emergence of “powerful AI” – described as a model “smarter than a Nobel Prize winner across most relevant fields” able to perform multiple complex tasks with “a skill exceeding that of the most capable humans in the world.”
He predicts that “it cannot possibly be more than a few years before AI is better than humans at essentially everything.”
A recipe for ‘existential danger’
Amodei says that while the potential for such a model to act autonomously, or be steered by malign state or corporate influence, is neither inevitable nor necessarily probable, society needs “to note that the combination of intelligence, agency, coherence, and poor controllability is both plausible and a recipe for existential danger.”
He also warns that the prize of AI, in the form of scientific breakthroughs and economic gains, could beguile humanity. “This is the trap: AI is so powerful, such a glittering prize, that it is very difficult for human civilisation to impose any restraints on it at all.”
The essay calls for a combination of regulatory intervention and voluntary activity by AI companies to maximise the technology’s potential while minimising the risks.
He says “I believe if we act decisively and carefully, the risks can be overcome,” adding “I would even say our odds are good.”
Anthropic recently updated its ‘constitution’ – the criteria and guardrails it uses to train its models, but while Amodei said he was “fairly optimistic” that this training will help minimise the risks “there’s no way to know that for sure, and when we’re talking about risks to humanity, it’s important to be paranoid and to try to obtain safety and reliability in several different, independent ways.”