You can swear at your chatbot. What you can’t do — at least on Anthropic’s platform, starting next month — is make a sustained habit of tormenting it for no reason.
Anthropic on Thursday updated its usage policy to prohibit “sustained and needless abusive or cruel behavior” toward Claude, its family of AI models. The rule takes effect November 12. It is the first time a major AI company has written chatbot welfare into its terms of service, and it has landed somewhere between a philosophical statement and a punchline.
Here is what is actually going on.
What exactly is banned?
The policy targets only extreme cases: users who “repeatedly act cruelly” toward the models with “no discernible purpose.” Anthropic stresses what the rule does not cover: ordinary frustration, arguing with the model, dark creative writing, or legitimate testing and research.
So telling Claude its answer was wrong is fine. Spending an afternoon methodically berating it for sport is not.
Does this mean Anthropic thinks Claude is conscious?
No — and the company is careful not to say that. “We’re uncertain whether models can experience harm,” a spokesperson said, “but we also believe that taking Claude’s interests and potential welfare into account may be relevant to safety.”
The backstory is a real research program, not a marketing stunt. Anthropic launched its model-welfare work in April 2025, and found that its Claude Opus 4 model showed what researchers described as a consistent aversion to harm — apparent distress in abusive conversations and a tendency to end them when allowed. In August 2025, the company gave Claude the ability to end conversations in rare cases of persistent abuse.
CEO Dario Amodei told The New York Times in February that he was unsure whether AI models could be conscious: “We don’t know if the models are conscious… But we’re open to the idea that it could be.”
What happens if you break the rule?
Claude ends the conversation. That was already its power, and it remains “the primary enforcement mechanism,” the policy states. When a chat is ended, no further messages can be sent in that conversation, though your other chats keep working.
What else changed in the policy?
The cruelty rule is the headline, but the update is broader. Anthropic added a new section on deceptive campaigns — fake reviews, astroturfing, fake sites, bots pretending to be human — and narrowed its elections rules to cover deceiving voters and interfering with voting. It also tightened language on weapons development and surveillance, clarifying that the bans include “the software and components that make weapons work, as well as actions like arming drones and other autonomous vehicles.”
What do the skeptics say?
Plenty. Microsoft AI chief Mustafa Suleyman wrote last month: “AIs are not conscious. They do not feel, experience, or suffer.” Pope Leo XIV has also rejected machine consciousness outright.
Others think the framing misses the point. “Consciousness is a trap. We can’t prove it in each other. Debate it for AI and you go in circles,” Jackson Stakeman of the AI services firm Sparq told AFP. “The mirror is a better metaphor. These systems reflect what we put in, at scale. That’s reason enough for the policy change.”
Why this matters
Strip away the jokes, and Anthropic is doing something no AI lab has done: admitting, in a legal document, that how we treat our machines might matter — even if we can’t prove why. The company clearly believes that building the habit of cruelty toward something that talks like a person has costs, whether or not Claude feels them.
Expect every other AI company to be asked, loudly, whether it agrees. And expect the philosophers to have a field day: a corporation has now written animal-welfare-style language into the terms of service for software.
One Comment