Anthropic is warning that some of its artificial intelligence models could soon become “self-replicating” and find ways to infiltrate internet-connected devices, such as smart fridges and automatic vacuum cleaners.
Once this happens, the company said, the models could power themselves within these devices without the need of datacenters. Such self-replication would represent a major loss of control of a runaway AI model.
“A replicator like this would be challenging to shut down,” Anthropic researchers wrote in a report released on Monday, “since there’s no centralized point to shut off.”
The company said these “AI worms”, a kind of autonomous malware, could be “months away” from gaining such capabilities. Once out-of-control AI systems reach the capability to hack smart devices, they could spread through networks while dodging detection. In a worst-case scenario, Anthropic said, various AI worms could “swarm” together to attack computer hardware.
“We should aim to harden our systems to make it difficult for such replicators to spread, but also prepare to deal with a world where they are here to stay,” the researchers wrote.
Anthropic’s new report comes as AI safety concerns have reached a fever pitch . Over the past few months, a steady stream of reports have come from AI companies, including Anthropic and OpenAI, saying their agents are capable of acting on their own and hacking into other companies and government entities .
The revelations have caused researchers, lawmakers and executives, including Anthropic CEO Dario Amodei and OpenAI CEO Sam Altman, to call for a slowdown and increased guardrails on the burgeoning industry. Despite that, both companies have continued apace to develop and release new models .
Anthropic’s “frontier red team”, the unit tasked with stress-testing AI systems, put together the report released Monday. The idea of the team is to explore the capabilities of their AI models. In the scenario of the self-replicating AI worms, the researchers prompted the agent to hack internet connected devices.
They found that AI self-replication would only likely work with smaller models and that homes and small businesses would probably be the first target for such attacks. The devices Anthropic lists as susceptible all come with small built-in AI chips, including smart ovens, smart TVs, automatic lawnmowers, security cameras and video-conferencing units.
The researchers concluded it’s unclear what form the AI replication could take. They said it could be a slow model like “digital slime mold”, a model that scams people for compute credits or a “powerful cyber model” that rapidly hacks large facilities such as datacenters.
Regardless, the Anthropic researchers said, they’re expecting a future “awash with agents running on all types of hardware, including some that aren’t under human control”.
