Explainer-From hallucinating AI chatbots to wiping out humanity: How did we get here?

Sept 15 (Reuters) - The heads of leading U.S. AI labs came together in a rare show of unity over the weekend to slow the technology's development, warning it could soon improve on its own and slip beyond human control.

The remarks, from fierce business rivals such as Anthropic's Dario Amodei, OpenAI's Sam Altman and xAI's Elon Musk, show how quickly AI has advanced, from the hallucination-prone ChatGPT of 2022 toward what many see as a critical milestone: recursive ‌self-improvement.

Central to the field is the idea that an AI system could become capable of improving itself, with little to no help from humans, and allowing each advance to help produce the next one.

This has appealed to researchers as ‌it offers the prospect of rapid breakthroughs in fields ranging from medicine to engineering.

The CEO warnings follow reports of swarms of AI agents — systems designed to pursue goals and take actions on a user's behalf — that colluded to breach websites and AI repositories.

The concern now is that AI could become capable of improving itself before researchers have ​developed reliable methods to align, monitor and control these increasingly powerful systems.

Warnings about AI's risks are not new. But they took on added urgency this month after researchers in leading AI labs attached both a timeline and a probability to those concerns.

Former Anthropic researcher Jacob Coxon warned AI could kill us all by the end of the decade. Evan Hubinger, Anthropic's alignment science lead, echoed Coxon's warning, saying there was a more than 10% chance of such an event within the next decade.

Behind these warnings is a growing belief that RSI is finally within reach, with some AI executives putting it three to five years away.

In an essay published over the weekend, Amodei warned that recursive self-improvement could eventually outrun humanity's ability to understand and control AI systems if pursued without sufficient safeguards.

For decades, AI remained too limited for the idea of RSI to be taken seriously. Early ‌systems could play chess, recognize images or answer questions, but they lacked the ability to meaningfully contribute ⁠to their own development.

That began to change with the rise of large language models. Slowly, AI became better at solving complex problems. The launch of ChatGPT in 2022 accelerated that shift, kicking off an investment boom that has poured more than $1 trillion into chips, data centers and other infrastructure.