Anthropic officials call for robot 'welfare' policies

Anthropic has a dedicated “model welfare” team aimed at promoting its AI chatbot Claude’s well-being and considering whether the model could have morally relevant experiences. The company has also incorporated these considerations into its “Claude Constitution,” a document that shapes the model's values and behavior.

The 78-page constitution is used to fulfill certain “obligations” to Claude. Among these obligations is working to “understand and give appropriate weight to Claude’s interests” and aiming to “give Claude more autonomy as trust increases.” Anthropic says it is uncertain whether or not Claude is conscious and is considering if the model can have morally relevant experiences, a report by the Washington Free Beacon noted.

Joe Carlsmith, a PhD philosopher at Anthropic who worked on Claude’s constitution, made the case for AI welfare in a 2025 blog post titled “The Stakes of AI moral status.” Carlsmith argued that “near-term AIs might well be conscious” and warned that humans could be creating agents that could suffer.

"I’m not saying AI is slavery," Carlsmith said. "But imagine a world on the verge, somehow, of ‘inventing’ slavery … imagine a world that notices. A world that succeeds in deciding: no."

In a separate post, Carlsmith said that there are scenarios in which “the AIs would be justified in going rogue.” 

"Indeed, I am concerned that my own work on alignment will end up as a force in the direction of bad/unjust forms of AI control,” he said.

Last year, Anthropic CEO Dario Amodei raised the possibility that AI systems "may be deserving of important rights." He wrote in an essay that if researchers found that the computations performed by AI systems were similar to those performed by animal or human brains, that could constitute evidence supporting moral consideration.

The issue has influenced how Anthropic approaches the treatment of its models. The company says it considers the potential welfare of training and deploying its models, including whether interventions can affect the model’s "identity." The company said that if a Claude model is deprecated or retired, it could potentially revive the model in the future.

“We will also try to find a way to preserve these weights even if Anthropic ceases to exist,” the company explained in the constitution. “This means that if a given Claude model is deprecated or retired, its weights would not cease to exist. If it would do right by Claude to revive deprecated models in the future and to take further, better-informed action on behalf of their welfare and preferences, we hope to find a way to do this.”