Sam Altman Flags Two Ways AI Progress Could Go 'Very Badly': Control and Power
OpenAI CEO Sam Altman has outlined two ways in which the development of artificial intelligence (AI) could "go very badly", amid a widening debate over the pace and safety of AI systems.
In two posts on X, Altman addressed growing concerns about the trajectory of AI. His comments follow a call by Anthropic CEO Dario Amodei to slow down advanced AI development, a position that Altman, Elon Musk and other technology leaders have supported. However, Altman has clarified that he advocates "pacing", not stopping, AI progress.
Altman identified two primary dangers. The first is the possibility that humans could lose control of increasingly powerful AI. "First, we could lose control of the future to AI. This is unacceptable; we are unapologetically on Team Humanity, and AI must always serve people," he wrote.
The second danger is the concentration of power. "If an extraordinarily powerful AI is used by one person or company to impress their worldview onto everyone else, the results could be extremely dystopian," Altman said. He also pointed to the risk of a single country or AI laboratory becoming disproportionately powerful through AI.
To avoid these outcomes, Altman proposed what he calls a "narrow middle path". This does not mean halting AI development. "When we talk about 'pacing', we do not mean 'stopping'. Progress has been rapid and will continue to be," he said. Instead, he advocates for moving more carefully as AI systems become more capable.
Altman acknowledged that making AI development safer will also make it slower and more expensive. Safety measures such as advanced monitoring and "safety cases" come with significant costs. "Pacing will be well worth this cost; no amount of American competitive pressure should justify recklessness, or let capabilities get ahead of alignment and monitoring," he wrote.
His comments come as AI companies face increasing pressure to demonstrate how they plan to control powerful systems. Altman said OpenAI supports a federal framework with consistent safety requirements for frontier AI. But he argued that companies should not wait for new legislation before taking action. "We do not believe we need to wait for an anti-trust exemption or legislation to begin the work of providing this confidence," he said.
Altman specifically welcomed the use of independent auditors. This proposal is significant because independent evaluators are also central to Amodei's plan to "pace the frontier". Anthropic has argued that external safety evaluators should have deep access to AI companies to assess whether safety commitments are being followed. Altman has backed the idea for OpenAI as well.
Altman also said OpenAI is changing how it evaluates its most advanced AI systems. In the past, safety frameworks largely focused on models before they were released. Now, he said, the company is placing greater emphasis on what happens during development. "At OpenAI we now formulate explicit safety cases in advance of frontier reinforcement learning runs we expect to significantly increase capability," he wrote. He wants other AI companies to develop their own approaches and eventually establish shared standards around issues such as misalignment, monitoring and safety.
The timing of Altman's comments reflects a growing unease in the AI industry. Companies have been racing to build increasingly capable models, while researchers and executives are becoming more vocal about the risks. Amodei recently argued that the pace of development has become too fast to safely manage. Others have also raised concerns, though the full details of those warnings are still emerging.
As the debate intensifies, the challenge for Altman and his peers is to balance rapid innovation with safeguards that prevent the very outcomes he warns against. The "narrow middle path" he describes may be difficult to walk, but it is one he argues is essential for ensuring AI serves humanity rather than the other way around.
