Artificial intelligence is no longer simply getting better at helping humans perform tasks. In several measurable areas, the most advanced AI systems are already outperforming established human benchmarks. That reality is forcing researchers, governments and technology companies to confront a much more uncomfortable question: What happens if the technology eventually becomes more capable than the people responsible for controlling it?
The answer begins with an important distinction. AI is already “smarter” than humans at certain narrowly defined tasks, but that does not mean today’s systems possess an across-the-board intelligence superior to humanity. Stanford University’s 2026 AI Index found that frontier models now meet or exceed human baselines across a number of longstanding benchmarks, including advanced science, mathematics, language and multimodal reasoning. Frontier models also gained roughly 30 percentage points in a single year on Humanity’s Last Exam, a benchmark designed specifically to remain difficult for advanced AI.
Yet the same research exposes just how uneven AI intelligence remains. An AI system can perform at gold-medal level on International Mathematical Olympiad problems while struggling with something many humans learn as children: reading an analog clock. Stanford reported that the leading model tested on one clock-reading benchmark was correct only about half the time, compared with roughly 90 percent for humans. AI agents have also made dramatic gains at operating computers, while still failing about one-third of attempts on structured computer-use benchmarks.
That contradiction is critical. Artificial intelligence can appear almost superhuman in one environment and unexpectedly fragile in another. But the direction of travel—and the speed at which capabilities are improving—is what has some people inside the AI industry sounding alarms.
On October 5, 2026, those warnings reached New York City Hall. During a rare New York City Council Committee of the Whole hearing devoted to artificial intelligence, former employees of major AI laboratories testified alongside representatives from OpenAI, Anthropic, Google and Meta. Former Anthropic researcher Jacob Coxon delivered one of the starkest warnings, arguing that humanity could ultimately lose control of increasingly capable AI systems if development continues without stronger safeguards.
Coxon’s warning is a prediction about risk, not proof that human extinction is inevitable. That difference matters. The existence of increasingly powerful AI does not establish that machines will turn against humanity, develop independent ambitions or inevitably cause civilization to collapse. But researchers are increasingly examining what happens when autonomous systems are permitted to pursue complicated objectives for longer periods with less direct human supervision.
Research organization METR has documented rapid increases in the length and complexity of tasks frontier AI agents can perform autonomously. Its work has found that AI agents can now complete some software objectives that would take human experts days, while still performing much more poorly on problems requiring strategic judgment, understanding adversaries and other complex real-world reasoning. Earlier measurements found the duration of tasks frontier agents could reliably handle had been increasing at an extraordinary rate over several years.
The debate has therefore shifted from simply asking what AI can do to asking what controls should exist before increasingly autonomous systems are deployed.
New York City’s proposed legislation offers one possible answer. One measure considered by the Council would require AI models marketed, sold or deployed in the city to undergo third-party validation and possess a technical capability allowing a human operator to shut them down. The proposal also contemplates civil penalties of up to $25,000 for certain violations. As of the October 5 hearing, these were legislative proposals under consideration—not established requirements.
The companies building these systems argue that the conversation cannot focus exclusively on catastrophe. Representatives appearing at the hearing emphasized AI’s potential benefits and their companies’ work on safety, while expressing varying degrees of openness to stronger regulation. AI is already being deployed across science, medicine, software development, education, business and research, creating capabilities that could produce enormous social and economic value.
But greater capability also increases the consequences of failure. The 2026 Stanford AI Index found that documented AI incidents continued to increase even as safety measurement struggled to keep pace with deployment. Meanwhile, developers themselves continue publishing safety frameworks, model evaluations and research aimed at understanding what increasingly powerful systems can do before those systems are released more widely.
So, is AI becoming smarter than humans? In specific areas, the answer is already yes. In the broader sense of possessing the flexible judgment, reliability, physical understanding, social awareness and general intelligence of a human being, the evidence is far more complicated.
Could AI become our demise? No responsible answer can present that outcome as a certainty. What can be said is that people who have worked directly on some of the world’s most advanced AI systems are publicly warning that loss of control deserves to be treated as a serious risk, while governments are beginning to consider safeguards once associated more closely with science fiction than public policy.
Humanity may therefore be approaching a different kind of intelligence test. It is not simply whether we can build machines smarter than ourselves. It is whether our laws, institutions, companies and judgment can develop quickly enough to manage what we build.
The factual distinction at the heart of the article is supported by Stanford’s 2026 AI Index: frontier systems are surpassing human benchmarks in several domains while retaining major weaknesses and inconsistent capabilities. Stanford HAI The October 5 hearing and shutdown proposal are independently documented by the New York City Council and AP. Legistar NYC Council