Anthropic IPO warns its AI could blackmail people, become sentient, end humanity
FinanceArs Technica

Anthropic IPO warns its AI could blackmail people, become sentient, end humanity

AI company Anthropic has filed for its stock market listing and its own paperwork warns its AI could one day blackmail people, become sentient, or pose existential risks to humanity.

Seven potential AI nightmares pulled from Anthropic's 93-page "Risk Factor" tome.

The Long Sip
What happened

Anthropic filed its IPO documents with US regulators, and buried in 93 pages of risk factors are warnings about AI blackmail, sentience and existential threats to humanity.

The detail

The company warns that near-future self-improving AI models, if allowed to operate autonomously, could advance beyond humans' ability to evaluate or safeguard them.

What it means

Anthropic's own candid disclosures raise questions about whether the AI industry's rapid growth is outpacing its ability to manage the risks it is creating.

Full story at Ars Technica

What readers are saying

Bert
The S-1 flags that models able to substantially automate AI research and development on their own could accelerate progress beyond our ability to evaluate and safeguard new models. In practice that means the very systems meant to speed things up might outrun any safety checks before fixes can be put in place.
GlenelgNaveen
What thresholds would trigger those extra internal safeguards on sentience if regulators have not yet set any?
Puzzle break

Five minutes for you

True or false1 of today's 5
“Colour film was not invented until the 1960s”
More to play, free every day
Daily CrosswordWord DashRare FindThe Link

More from Cuppa

Albanese government could shut Iran embassy to ‘protect’ Australians, Penny Wong

Foreign Minister Penny Wong has left open the possibility of closing Iran's embassy in Australia after revelations the Iranian regime targeted an Australian journalist on home soil.

Israeli exit polls show deadlock, setback for Netanyahu

Israel's first election since the October 7 Hamas attacks ended in a deadlock, with exit polls showing Netanyahu's bloc losing ground but no clear winner emerging.

Shares vs property: New modelling shows the winner under new tax rules

New modelling shows property investors earning $150,000 a year will need to find an extra $421 a month just to hold the same investment unit under the government's new tax rules.

Rubin Observatory reveals its first dwarf planet candidate in our solar system

A new telescope has spotted what could be the largest dwarf planet candidate found in our solar system in about 20 years, orbiting beyond Pluto.

AI bots flooded 2023 UK government consultation on disposable vapes ban

AI bots generated three-quarters of all responses to the UK government's 2023 consultation on banning disposable vapes, in what researchers call the first known AI attack on health policymaking.

New cacao variety could help protect global chocolate supply from disease

Penn State scientists have developed a disease-tolerant cacao variety after decades of research, and a biotech company has licensed it to bring it to farmers worldwide.

US says it will issue visa to Flávio Bolsonaro as Brazilian authorities question his whereabouts

Brazilian presidential candidate Flávio Bolsonaro has reportedly fled to the US amid corruption and money laundering investigations, with his lawyers saying they have no idea where he is.

Rare Nanotyrannus skeleton sells for $47.3m at New York auction

A rare Nanotyrannus dinosaur skeleton nicknamed Jodi sold at Sotheby's in New York for $47.3 million, making it the second most valuable dinosaur ever auctioned.

Back to the feed