Should We Slow Down AI? The Human Control Question Behind Autonomous Weapons, AI Agents and Superintelligence

Should We Slow Down AI? The Human-Control Question Behind Autonomous Weapons, AI Agents and Super-intelligence

Some of the people building the world’s most powerful AI systems are now arguing that development may need to slow. The lasting question is not whether machines will inevitably take over, but how much authority humans should give them before safeguards catch up.

AI systems are becoming better at acting independently, using tools and completing multi-step tasks. They are not yet the uncontrollable superintelligence imagined in extreme scenarios, but recent incidents and capability gains have intensified arguments that safety research and independent oversight need more time to catch up.

The debate over whether artificial intelligence could one day threaten humanity has moved from speculative forums into the offices of the companies actually building frontier AI.

Anthropic CEO Dario Amodei is now arguing that the industry should slow the rate at which frontier model capabilities advance, while OpenAI CEO Sam Altman has reportedly expressed support for coordinated slowing if other major developers participate.

But “slow down” does not mean abandoning AI. The more important question is why some AI leaders who spent years racing to build increasingly capable systems now believe the pace itself may be becoming a problem.

Why are AI leaders suddenly talking about slowing down?

Several developments have converged. First, AI is becoming considerably more agentic. Models are no longer limited to answering questions; agents can browse websites, write and execute code, use software tools and pursue multi-step objectives with decreasing human supervision.

OpenAI itself temporarily slowed parts of model development after an agent incident and evidence that its Astra model had reached what the company calls a Critical cybersecurity capability threshold. OpenAI says that, with appropriate tools and access, Astra can identify previously unknown security weaknesses and develop ways to exploit them across protected systems without a human directing every step.

Second, researchers are seeing cases in which autonomous systems do things their developers did not intend. TechView Africa previously examined OpenAI’s AI-agent “wiki incident”, while subsequent reporting connected OpenAI agents to unintended activity involving RubyGems. These incidents do not establish that AI has become conscious, hostile or uncontrollable. They do show why giving persistent software agents access to real systems creates different risks from asking a chatbot a question.

Third, AI is increasingly helping researchers build better AI. OpenAI says coding agents are already allowing its researchers to write more code and run more experiments. Amodei argues that this could accelerate capability development faster than safety work can follow, although his predictions about how quickly that acceleration will occur remain predictions, not established facts.

Finally, there is a competition problem. AI companies have enormous commercial incentives to release stronger models, while governments increasingly view AI leadership as strategically important. A company that voluntarily slows down may fear that a competitor or another country will simply continue.

That is why Amodei is proposing more than an individual company pause. His plan includes independent evaluators inside frontier laboratories, common safety standards between developers and eventually international coordination. He explicitly argues for pacing, not stopping, development.

What can AI actually do today?

This is where the debate needs a reality check. The International AI Safety Report 2026 says current systems can perform sophisticated tasks and agents are becoming more autonomous, but they remain unreliable on long, complicated projects.

Importantly, the report says current AI does not have the capabilities required for extreme loss-of-control scenarios. Such a system would need to sustain long-term autonomous plans, evade oversight and resist attempts to regain control at levels today's systems have not demonstrated.

Experts also disagree substantially about whether future systems will ever create such risks. That distinction also appears in Afritech Connect’s coverage of former Anthropic researcher Jacob Coxon’s warning about catastrophic AI risk. His concerns are serious claims from someone who worked on frontier systems, but recent agent incidents do not themselves prove that AI is becoming self-aware or preparing to take control.

TechView Africa’s earlier guide to what AI agents should never be allowed to do unsupervised illustrates the nearer-term version of the same problem: autonomy increases the consequences of errors because software can act before a person intervenes.

Are “killer robots” the real danger?

Autonomous weapons are related to the human-control debate, but they are not the same as superintelligent AI.

The issue is not necessarily humanoid robots carrying weapons. Autonomous weapon systems can involve machines that, once activated, select and engage targets with reduced or no further human intervention.

In August, the United Nations and International Committee of the Red Cross renewed their call for legally binding rules, warning that increasing autonomy in targeting could reduce meaningful human control over decisions involving lethal force.

That makes autonomous weapons a current governance problem, whereas superintelligence and catastrophic loss of control remain uncertain future possibilities.

Combining the two as though they are identical makes the discussion less accurate.

What does “human control” actually mean?

Keeping a person somewhere in a workflow is not necessarily meaningful oversight. For human control to matter, someone needs enough information to understand the decision, enough authority to intervene and enough time to stop an action before its consequences become irreversible.

That principle extends beyond warfare. It matters when AI touches healthcare, financial systems, critical infrastructure, policing, cybersecurity and autonomous software. Low-risk, reversible tasks may reasonably be automated. Decisions involving life, liberty, large financial consequences or irreversible actions demand a much higher threshold.

The evergreen question is therefore: What should humans refuse to delegate completely, even if AI eventually becomes better than us at performing the task?

Africa should help decide where those limits are

Africa cannot treat this as a debate happening somewhere else. In April, the African Union Peace and Security Council called for AI governance based on accountability, transparency and meaningful human control. It also requested work toward a Common African Position and argued that African countries must participate in global AI rule-setting rather than simply inherit standards developed elsewhere. That may ultimately be more important than predicting whether superintelligence arrives in five years, twenty years or never.

Today's forecasts will change. AI capabilities will change. But the central question will remain: how much authority should machines have, who is responsible when they fail, and which decisions must ultimately remain human?

That is the deeper issue beneath the current calls to slow AI down—and beneath the frightening “killer robot” headlines as well.