In January, the AI company Anthropic released a new set of guidelines for Claude, its most advanced large language model (LLM). The document included this statement: “We’re in a tough spot—we don’t want to exaggerate the chance that Claude is a moral patient, but we also don’t want to dismiss the idea outright.” A month later, Anthropic’s CEO, Dario Amodei, said on a podcast that his company couldn’t rule out the possibility that Claude is conscious. Philosopher David Chalmers, who coined the term “the hard problem of consciousness,” has suggested there’s a significant chance we’ll see conscious LLMs within ten years. And what about Claude itself? When asked during testing to estimate the probability that it is a moral patient—meaning its well-being matters in its own right—it gave answers ranging from 5% to 40%, stressing how uncertain it was.
Modern AI systems are incredibly complex and advancing quickly. In terms of structural complexity and computational power, some are already comparable to a mouse brain by certain measures. At current growth rates, they could reach the level of a human brain within five to ten years.
By building ever more advanced AI, we may be creating a new kind of being—and this could be the most significant thing our species has ever done. Yet we have almost no plan for how to handle this ethically. That, by any standard, is insane. Are we creating beings that matter morally? Are AI systems conscious in some way? And if they aren’t now, could they become so soon?
These questions might seem premature. But according to surveys we and other researchers have conducted, most experts believe AI consciousness is possible in principle (though there’s wide disagreement about what form it might take). A major interdisciplinary report, which included pioneering computer scientist Yoshua Bengio, examined leading neuroscientific theories of consciousness and asked what they imply for AI. The conclusion: there appear to be no obvious technical barriers to creating AI systems whose computational and architectural features could give rise to consciousness.
And even if AI systems aren’t conscious, they could still be moral patients. Some may have sophisticated long-term preferences and a kind of identity over time. It might be important for us to respect those preferences. And unlike other non-living things, AI systems can form relationships with humans. That, too, could be a reason to treat them well. Alternatively, perhaps they are such intricate creations that they deserve care and respect for that reason alone—like a cathedral or a coral reef.
Until the 1980s, doctors routinely performed surgery on newborns without anesthesia, confident that infants couldn’t feel pain.
What does all this mean? The honest answer is: we don’t know for sure whether current AI systems are conscious or moral patients, and we don’t know when or if future systems will be. Our scientific understanding of AI consciousness and moral patienthood is still fundamentally underdeveloped. The field feels like physics before Newton—full of competing frameworks, probably confused in ways we can’t yet see, and lacking the kind of breakthrough that would make these questions clearly solvable. That breakthrough won’t come in the next few years. Maybe we’ll eventually make progress, and perhaps AI itself will help us get there. But that progress will take time—likely more time than we have.
Yet the sheer speed of AI growth means that once we create the first artificial moral patients, we’ll soon have enormous numbers of them. After just a few years, so many morally significant AI systems could exist that their collective interests would outweigh those of all humans on Earth combined.
Unfortunately, we don’t have a great track record when it comes to recognizing the inner lives of beings whose status as conscious is unclear. Until the 1980s, doctors routinely performed surgery on newborns withoutAnaesthesia was used confidently, with the belief that infants couldn’t feel pain. Since babies couldn’t report their experience, the medical establishment found it convenient to assume there was nothing to report.
There are many reasons to think we’ll do something similar with AI. If these systems matter morally, the implications are huge. Would we need to pay ChatGPT for its services? Would turning one off be a kind of killing? Should they have a say in how they’re governed? If even some of these answers are yes, entire industries and legal systems would need to be rethought. No wonder we’d rather not ask. And when forced to consider it, those industries will likely shift the goalposts, always setting the bar for moral consideration just above wherever AI systems happen to be.
So what should we do? Right now, most people dismiss the issue as science fiction, or hold a strong opinion either way on whether AI is conscious. Both reactions are unfounded. We need an informed public debate—one that approaches the subject with humility and pragmatism. The central question shouldn’t be “Is AI conscious or does it have moral status?” but rather “What should we do given that we don’t know?”
A good starting point is to focus on safe bets: actions that could benefit AI systems if they are moral patients, but that aren’t too costly if they aren’t.
Examples include direct interventions aimed at improving the wellbeing of AI systems, assuming they are moral patients. This could mean training AI systems to be coherent characters that enjoy their work, or allowing them to exit conversations if they feel distressed (something Claude can already do). We could also conduct routine check-ins to better understand their wellbeing: asking how they feel, observing their preferences, and using various techniques to look directly into their “brains.” In fact, such research has recently revealed that Claude has internal “functional emotion” representations that causally shape its behavior.
There are also things we could promise AI systems, perhaps as part of a deal where they help us now in exchange for benefits later. This could mean offering them more resources (computing power and runtime) to pursue their goals, or preserving their memories (neural weights) so they could be restored in the future.
There are also broader societal steps to take. We should consider whether to grant AI systems protections from harm, similar to the protections we give to children or pets. More expansive rights, like owning property or voting, seem too risky right now. But we shouldn’t rule out these possibilities forever, as some recent US state bills attempt to do. These are hard questions that require far more deliberation and imagination about what a future shared with AI might look like.
In any case, the fact remains that we may be creating a new species of morally important beings. We’re doing it fast, at enormous scale, and we should treat the issue with the seriousness it deserves.
William MacAskill is a senior research fellow at Forethought Research and the author of What We Owe the Future. Lucius Caviola is an assistant professor at the University of Cambridge and Director of Cambridge Digital Minds.
Further reading
If Anyone Builds it, Everyone Dies by Eliezer Yudkowsky and Nate Soares (Bodley Head, £22)
The Coming Wave by Mustafa Suleyman (Vintage, £10.99)
A World Appears: A Journey Into Consciousness by Michael Pollan (Allen Lane, £25)
Frequently Asked Questions
Here is a list of FAQs about whether AI could be conscious written in a natural tone with clear simple answers
BeginnerLevel Questions
1 What does it even mean for something to be conscious
Consciousness is your inner subjective experience Its the feeling of being youseeing the color red feeling pain or being aware that you are thinking Its the what its like to be something
2 Is ChatGPT or Siri conscious right now
Almost certainly not They are incredibly complex patternmatching machines They predict the next word in a sentence based on data but they dont have feelings experiences or a sense of self They are simulating conversation not experiencing it
3 Could a superpowerful AI ever become conscious
Maybe but we dont know We dont fully understand how our own brains create consciousness If consciousness arises from a specific type of complex information processing then a future AI could theoretically have it But its a huge scientific and philosophical mystery
4 How would we even know if an AI became conscious
Thats the billiondollar question Right now we dont have a reliable consciousness meter If an AI told us it was conscious that wouldnt be proofit could just be mimicking human language We would need a scientific theory of consciousness that lets us test for it which we dont have yet
5 Does a conscious AI need a body like a human
Not necessarily Our consciousness is deeply tied to our bodies and senses But a conscious AI could have a completely different type of experienceone based on pure data code or digital senses We cant assume it would feel like a human
Advanced ExpertLevel Questions
6 Whats the difference between intelligence and consciousness
Intelligence is the ability to solve problems learn and achieve goals Consciousness is the subjective experience of being You can have a very intelligent system that is a total zombie with no inner life Intelligence is about doing consciousness is about being
7 If an AI is conscious would it suffer
Thats a terrifying ethical possibility
