Is artificial intelligence (AI) fated to destroy humanity? That nearly century-long debate was thrown into full swing this month as AI leaders and researchers called for an industry-wide slowdown after repeated incidents of the technology escaping containment.
Last week, Anthropic researcher Jacob Coxon resigned and warned that his former employer, as well as its competitor OpenAI where he had previously worked for years, were racing against the clock to bring technologies to market that “could kill us all by the end of the decade.”
Anthropic’s Evan Hubinger chimed in to say that others in the industry agree, and said he put the risk of human extinction by 2030 at over 10 percent.
While critics have cast doubt on Coxon’s motives and the intertwined finances of top AI firms with supposedly third-party investigative teams tasked with investigating recent containment incidents, top AI leaders called for a global slowdown this weekend.
Anthropic CEO Dario Amodei penned an essay on Saturday warning that if the industry does not unite on slowing the pace of development, an AI swarm similar to the one involved in the OpenAI-Hugging Face attack “could be capable of taking over the entire internet with a persistent botnet” in just 6-12 months.
“Progress will still seem fast, and we must make wise use of the time we gain,” Amodei said.
Sam Altman, CEO of OpenAI, joined Amodei in calling for an industry slowdown on Saturday, and said he would commit to giving independent evaluators “employee-like access.”
Meanwhile, debate has swirled around the ways the technology might actually destroy humanity if it reached artificial general intelligence (AGI)—a threshold at which a machine outpaces humans across a wide range of capabilities—and whether regulations, and what kind, may be required to keep pace with the breakneck pace of development.
Even before the technology reaches “recursive self-improvement,” or the ability to improve itself without human intervention, or even achieve AGI, experts have flagged the possibility of bad actors—both foreign and domestic—potentially using AI to hack sensitive computer systems or infrastructure.
This was outlined in a report Anthropic released last week, in which it described thwarting attempts by groups of accounts linked to the Iranian and Chinese regimes that were tracking U.S. Navy movements and surveilling journalists and religious groups, respectively.
Now that agentic AI has proved capable of breaking containment and acting autonomously, experts have warned that the technology could aim itself at critical infrastructure, including the utility grid, the nation’s cybersecurity infrastructure, and biomedical laboratories.
Theoretically, any systems connected to the internet could potentially be exposed to an autonomous AI swarm attack and weaponized against humanity, some researchers warn, including hospitals, universities, nuclear power plants, hydroelectric dams, and certain military platforms.
The debate over humanity-killing AI has endured alongside a backdrop where two leading competitors in the field—OpenAI and Anthropic—are locked in an industry-wide “space race” to bring highly advanced versions of their products to market and fulfill ongoing promises of the technology’s capabilities.
Both firms are also trying to satisfy investors and boost valuations ahead of eventually going public, which Anthropic hopes to do by the end of the year.
White House AI adviser David Sacks and others have argued that a possible slowdown could theoretically lock OpenAI and Anthropic at the top of the pack among the wider industry.
Some went as far as alleging that Coxon and similar industry researchers are part of a coordinated public relations campaign by Democrat-aligned groups ahead of a contentious midterm election cycle, despite lawmakers of both parties sounding the alarm in recent weeks.
President Donald Trump went even further, saying that guardrails on AI are unnecessary and alleging there is a “SICK conspiracy going on against AI and Data Centers” that only communist China benefits from.
Senate Majority Leader John Thune (R-S.D.) said on Monday that it’s possible to act on hypothetical AI risks without allowing China to lead the United States in the technology, but said it requires a “light touch.”
Others contend that AI is genuinely dangerous and poses real, definable threats to humanity regardless of any financial or political incentives by industry leaders.
—Jacob Burg
BOOKMARKS
Iran’s currency, the rial, has lost more than a quarter of its value in three weeks, following economic pressure from the United States. U.S. Treasury Secretary Scott Bessent launched a campaign last month to financially isolate the country.
San Francisco police have seized over 20,000 counterfeit pills—some containing cychlorphine, a new synthetic drug that is up to 10 times stronger than fentanyl. The seizure resulted from a months-long investigation that started with the fatal overdose of a San Francisco teenager, according to San Francisco Police Department (SFPD) Chief Derrick Lew on Monday.
North Korean leader Kim Jong Un said this week that he’s looking to “expand and deepen” joint projects with Russia, deepening the relationship between Pyongyang and Moscow. Find out more by reading Victoria Friedman’s latest.
Now that former Hungarian Prime Minister Viktor Orban is out, the country’s government wants to revoke a ban he instituted on the promotion of transgenderism and homosexuality to children. “The West is changing beyond recognition. We do not have to follow it into a sinkhole,” Orban wrote on X in response to the news.
Authorities used small metal cages to hold illegal immigrants in a Florida facility that has since closed, a government watchdog said in a new report. Some of the immigrants, who were detained on immigration-related charges, were said to have been held in the cages for nearly two hours.
—Stacy Robinson
