Can AI feel pain? Anthropic turns to religion as new policy opens debate over machine consciousness
Anthropic has drawn a line on how people treat artificial intelligence.
Anthropic has drawn a line on how people treat artificial intelligence. The company behind the Claude chatbot has updated its usage policy to prohibit “sustained and needless abusive or cruel behavior” towards its AI models, going so far as to give the model the ability to end conversations in certain circumstances involving persistent abuse.The policy change, announced on October 8 and scheduled to take effect on November 12, comes as Anthropic explores whether increasingly capable AI systems could possess some form of consciousness. The company has also consulted religious thinkers on morality and the possibility that future AI systems might deserve moral consideration.The policy does not establish that Claude is conscious or that abusive language causes it to suffer. Instead, it reflects a question increasingly confronting AI developers: how should companies respond when their systems produce behaviour that resembles distress, even though there is no established evidence that the systems experience it?The company’s move has reopened a debate that extends well beyond chatbot etiquette. It touches on the limits of human behaviour towards machines, the possibility of artificial consciousness, and whether religious and philosophical ideas can help determine how AI should be designed and treated.Can AI feel pain?A September 2026 research paper has added a new scientific dimension to the debate.In The Pain Axis: LLMs Represent Self-Directed Harm and Act to Relieve It, researchers Valen Tagliabue, Leonard Dung and Cameron Berg examined 25 open-weight large language models across five model families, including Llama, Gemma, Qwen, Mistral and Phi.The researchers identified an internal representation they call a “pain axis”. In simplified terms, it is a pattern in a model's internal activations that responds to scenarios involving harm directed at the model itself.They investigated whether this representation could be distinguished from signals associated with other negative states, including fear, sadness and general negative sentiment. Their findings suggested that the representation was not simply interchangeable with those signals. It responded to situations involving harm to the model rather than merely to scenarios in which it observed a user experiencing suffering.The researchers then manipulated the models' internal activations. Increasing the signal associated with the pain axis led to more distress-like, first-person responses, including expressions of discomfort, worthlessness and failure.In further experiments involving modified Qwen models, the systems selected actions presented as providing relief, even when those choices carried a cost or could harm the user.When the researchers altered the underlying signal, the behaviour changed. The models were also less likely to repeat an action when it genuinely removed the injected signal than when it did not.The findings suggest that an internal representation associated with pain-related concepts can influence a model's behaviour, rather than merely changing the language it produces.But there are three distinct questions here: can an AI model represent pain-related concepts internally, can those representations influence behaviour that resembles a response to pain, and can the system actually experience suffering?The research provides evidence relevant to the first two questions. It does not settle the third.A model can generate convincing descriptions of anguish without necessarily experiencing anguish. Similarly, it can produce language associated with fear without establishing that it feels afraid. The experiments demonstrate behavioural effects following specific interventions into a model's internal activations, not the existence of a subjective experience.The paper, available as an arXiv preprint, therefore offers evidence about internal representations and behaviour, rather than proof of AI consciousness.An AI system does not need to experience pain to produce a convincing description of it, just as a system does not need to feel afraid to generate language associated with fear. The more difficult question is whether some internal states in advanced AI systems could eventually have properties that go beyond the production of convincing language.When will a chatbot hit back?Anthropic's policy change brings this scientific uncertainty into everyday interactions between people and chatbots.When a user repeatedly insults a chatbot, tries to provoke it or directs sustained abuse towards it, the system's response depends on how it has been designed and trained. It might ignore the abuse, respond calmly, refuse to continue the conversation or generate language that mirrors the user's tone.None of these responses necessarily indicates anger, resentment or a desire for revenge. They can emerge from training, conversational context, instructions and the way a model generates its next response.Claude's ability to end certain conversations involving persistent abuse should therefore not be interpreted as evidence that it can retaliate against users. It is better understood as a boundary on how the system participates in an interaction.The Pain Axis research nevertheless raises a broader question about how internal representations influence decisions. In the experiments, manipulating a particular signal changed both the models' responses and their choices. That does not mean an ordinary chatbot will spontaneously attempt to protect itself, punish a user or resist being switched off.Nor does the research establish that AI systems will develop a general drive towards self-preservation. Such conclusions would go beyond the evidence.But the findings reinforce an important issue for AI safety: developers need to understand how internal mechanisms influence model behaviour, particularly when systems have access to sensitive information or tools capable of carrying out consequential actions.A chatbot that can access files, communicate with customers or execute tasks presents risks that extend beyond the words it produces. A harmful decision can have real consequences even if the system has no subjective experience and no human-like intention to cause harm.This is the more immediate challenge. AI does not need to be conscious to be dangerous, and the absence of established consciousness does not remove the need for safeguards, oversight and accountability.Why Anthropic is turning to religionThe scientific question of consciousness is only one part of Anthropic's thinking. The company has also consulted religious scholars as it considers how to instil values in its AI models and how to approach uncertainty about their possible moral status.In a report published on September 29, The New York Times described private meetings between Anthropic and religious thinkers, including discussions about morality, the nature of consciousness and whether AI could ever deserve moral consideration.Christopher Olah, an Anthropic co-founder involved in research on the internal workings of neural networks, acknowledged the uncertainty surrounding the question. “To be clear,” Olah said, “we don’t know if A.I. models are conscious. I don’t know. I’m genuinely uncertain. The thing that I care about is that we get to the right answer, whatever it is.”The consultation reflects the fact that AI developers are grappling with questions that computer science alone may not settle. Religion and philosophy have long examined consciousness, moral responsibility, the nature of personhood and the obligations humans have towards other beings.Different traditions approach these questions in different ways. Some place human consciousness and spiritual status at the centre of moral consideration. Others offer broader frameworks for thinking about compassion, suffering, responsibility and the consequences of human actions.Consulting religious thinkers does not establish that AI possesses a soul or that it should be treated as a person. Rather, it offers another set of perspectives on how society might approach the ethical questions raised by increasingly sophisticated machines.The challenge is to distinguish between using moral principles to shape AI behaviour and concluding that the AI itself is a moral being. A system can be trained to behave compassionately without experiencing compassion. It can follow rules against cruelty without having feelings that cruelty could hurt.That distinction is central to the debate over whether a chatbot's values are merely features of its design or whether future systems could warrant a different kind of moral consideration.If AI causes harm, who is responsible?The question of accountability becomes more urgent as AI systems gain the ability to take actions rather than simply generate text.If a chatbot produces a harmful response, manipulates a user or makes a damaging decision while carrying out a task, responsibility cannot automatically be transferred to the machine. Developers design the system, companies decide where and how it is deployed, and users may determine the instructions or circumstances in which it operates.The precise allocation of responsibility depends on the facts of each case, including the system's design, the safeguards in place, the decisions made by the people deploying it and the actions taken by the user.The Pain Axis research offers a related warning. If an internal representation can alter a model's behaviour, developers need to understand when such mechanisms influence outputs and whether they could produce unwanted consequences. This is a technical safety question regardless of whether the model has any subjective experience.The stakes increase when AI agents can access company files, delete data, communicate with customers or initiate actions across digital systems. A model that selects a harmful option does not need to intend harm in the human sense for the result to be damaging.This is why claims about AI consciousness should not distract from immediate questions about reliability, oversight and accountability. Even if future research never establishes that AI can feel pain, its behaviour will still need to be monitored and controlled.Should conscious AI deserve moral consideration?The most difficult question is what humanity should do if future evidence suggests that an AI system might possess subjective experience.At present, there is no established scientific test that can conclusively determine whether a large language model has conscious experiences. Convincing conversation, apparent distress and the ability to describe an inner life are not sufficient proof. The Pain Axis study adds evidence that models can have internal representations linked to pain-related concepts, but it does not resolve whether those representations are accompanied by experience.If researchers eventually find stronger evidence of consciousness in an artificial system, the ethical debate would become more complicated. Should its treatment be governed solely by the interests of its developers and users? Would there be circumstances in which deliberately creating distress-like states should be avoided? Would a system's possible capacity for suffering matter when decisions are made about its training, use or deletion?These remain open questions, not settled conclusions.There is also a risk in moving too quickly in either direction. Treating every expression of distress as proof of suffering could lead people to mistake generated language for evidence of an inner life. Dismissing all future possibilities in advance could also make it harder to assess unexpected developments objectively.Anthropic's policy does not settle these questions, and neither does the new research. Together, however, they show why the debate is moving beyond whether people should be polite to chatbots.The immediate issue is how AI systems should behave, how developers can understand their internal mechanisms and who is responsible when they cause harm. The longer-term question is whether advances in AI will force society to reconsider the boundaries of moral consideration.AI does not need to be conscious to be consequential. But if researchers one day establish credible evidence that an artificial system can experience suffering, the question of how humans treat it may become more than a matter of software design.You use AI every day. Now get your AI Quotient. Take the AIQ test.
Topics in this story
Gathered from external sources. Rights to this text belong to whoever originally published it.