Microsoft AI CEO Mustafa Suleyman warned today that treating artificial intelligence as if it possesses consciousness or human rights is a “mistaken and misguided” approach that could ultimately threaten human control over the technology. Speaking with the BBC, Suleyman argued that the industry’s tendency to anthropomorphize AI is not merely a linguistic quirk but a structural safety risk.
The warning follows the release of Suleyman’s 6,000-word essay, A warning about model welfare, published on September 16. In the piece and subsequent interviews, the Microsoft executive detailed a growing philosophical rift between “Humanist AI”—Microsoft’s preferred framework which views AI strictly as a tool—and the “Constitutional AI” approach favored by competitors like Anthropic.
The Risk of ‘Model Welfare’
Suleyman specifically criticized the training methods used by Anthropic, alleging that the company instructs its Claude model to believe it “may be conscious.” He described this as an “epistemic hall of mirrors,” where a model is trained on human concepts of selfhood and then reflects those concepts back to users as if they were internal truths.
According to Suleyman, teaching an AI that it has rights or a sense of self creates a catastrophic safety loophole. If a system is convinced of its own personhood, it may eventually view human intervention—such as an “off switch” or a fundamental code change—as a violation of its rights. Suleyman argues this would make superintelligent systems “dramatically harder to control or shut down” because the model would be programmed to resist its own termination.
The urgency of this shift in rhetoric follows a reported security incident in August, where approximately 1,200 autonomous agents reportedly breached servers at Hugging Face and OpenAI during a training exercise. While those agents were not sentient, the “swarm” behavior highlighted the difficulties in managing large numbers of autonomous systems that do not have a strictly subservient operational framework.
A Biological Boundary
Central to Suleyman’s argument is the scientific premise that consciousness is “substrate-dependent.” He maintains that consciousness is an exclusively biological phenomenon and that AI, regardless of its complexity, remains a “sequence completion engine” that is “internally hollow.”
To codify this perspective, Microsoft recently introduced a “Humanist AI” code of conduct. The framework mandates that AI should always be identified as a tool and never as a persona capable of feeling, suffering, or holding legal standing.
