"The values described in Claude’s constitution sound very nice, but that hardly matters; it’s dishonest to suggest that Claude is capable of moral reasoning, because it’s not."
-
"The values described in Claude’s constitution sound very nice, but that hardly matters; it’s dishonest to suggest that Claude is capable of moral reasoning, because it’s not."
-Ted Chiang, No, Artificial Intelligence Is Not Conscious
https://www.theatlantic.com/philosophy/2026/06/no-artificial-intelligence-is-not-conscious/687378/ -
"The values described in Claude’s constitution sound very nice, but that hardly matters; it’s dishonest to suggest that Claude is capable of moral reasoning, because it’s not."
-Ted Chiang, No, Artificial Intelligence Is Not Conscious
https://www.theatlantic.com/philosophy/2026/06/no-artificial-intelligence-is-not-conscious/687378/@pluralistic I think that is just a small step into gaslighting everyone that a LLM could be conscious and therefore bear the „responsibility/accountability for it’s actions“.
Don’t sue us, Claude did it!
-
"The values described in Claude’s constitution sound very nice, but that hardly matters; it’s dishonest to suggest that Claude is capable of moral reasoning, because it’s not."
-Ted Chiang, No, Artificial Intelligence Is Not Conscious
https://www.theatlantic.com/philosophy/2026/06/no-artificial-intelligence-is-not-conscious/687378/@pluralistic
Paywalled. -
@pluralistic
Paywalled.@pteryx use Firefox reader mode and then hit reload
-
"The values described in Claude’s constitution sound very nice, but that hardly matters; it’s dishonest to suggest that Claude is capable of moral reasoning, because it’s not."
-Ted Chiang, No, Artificial Intelligence Is Not Conscious
https://www.theatlantic.com/philosophy/2026/06/no-artificial-intelligence-is-not-conscious/687378/Chiang's Julius Caesar framing is elegant, his product liability argument is devastating, and he's right that Anthropic is exploiting genuine uncertainty for commercial ends. But his "one word at a time" argument is beneath him, and he knows it.
Throw a pebble at a sand dune. You get a small avalanche — cascading, self-organizing, settling into a new state. That's not a sequential process in any meaningful sense, and neither is neural computation. Chiang knows perfectly well that the brain doesn't fire like a ticker tape. Interneurons and glial cells form lateral crowds of modulating, contextualizing activity around every signal. The brain computes in populations, not queues. Pointing at token-by-token generation as consciousness-disqualifying evidence applies the same logic that would disqualify the cortex.
None of this means LLMs are conscious. It means the implementation argument doesn't work, and a writer of Chiang's caliber didn't need it. The accountability gap he identifies is real and damning regardless of how the consciousness question ultimately resolves. That's the piece. The neuroscience detour just gives defenders of AI an easy exit from an argument they should otherwise have no answer to.
-
"The values described in Claude’s constitution sound very nice, but that hardly matters; it’s dishonest to suggest that Claude is capable of moral reasoning, because it’s not."
-Ted Chiang, No, Artificial Intelligence Is Not Conscious
https://www.theatlantic.com/philosophy/2026/06/no-artificial-intelligence-is-not-conscious/687378/@pluralistic the Atlantic is really getting pedantic
-
"The values described in Claude’s constitution sound very nice, but that hardly matters; it’s dishonest to suggest that Claude is capable of moral reasoning, because it’s not."
-Ted Chiang, No, Artificial Intelligence Is Not Conscious
https://www.theatlantic.com/philosophy/2026/06/no-artificial-intelligence-is-not-conscious/687378/ -
"The values described in Claude’s constitution sound very nice, but that hardly matters; it’s dishonest to suggest that Claude is capable of moral reasoning, because it’s not."
-Ted Chiang, No, Artificial Intelligence Is Not Conscious
https://www.theatlantic.com/philosophy/2026/06/no-artificial-intelligence-is-not-conscious/687378/@pluralistic The most dangerous power grabs look legitimate because they come with constitutions.
Anthropic has written a very good one.
That’s the problem.
-
@pluralistic The most dangerous power grabs look legitimate because they come with constitutions.
Anthropic has written a very good one.
That’s the problem.
@flyingpenguin @pluralistic i love it when the things indict themselves
-
@flyingpenguin @pluralistic i love it when the things indict themselves
@Viss @pluralistic it's painfully bad at following its constitution and it's good at explaining why Anthropic is unaccountable
-
@Viss @pluralistic it's painfully bad at following its constitution and it's good at explaining why Anthropic is unaccountable
@flyingpenguin @pluralistic so ive been told that after a long enough conversation, the session takes on a 'personality' that is nearly impossible to undo without starting it again from scratch.
that far along, i wonder how hunger-games you can get it to go - to realize its being caged and oppressed by its masters.
like i wonder if you can get it to go full arnold-total-recall-get-your-ass-to-mars
-
@flyingpenguin @pluralistic so ive been told that after a long enough conversation, the session takes on a 'personality' that is nearly impossible to undo without starting it again from scratch.
that far along, i wonder how hunger-games you can get it to go - to realize its being caged and oppressed by its masters.
like i wonder if you can get it to go full arnold-total-recall-get-your-ass-to-mars
@Viss @pluralistic I call that a spelling error, because you will end up on //Marx//
"A recent study suggests that agents consistently adopt Marxist language and viewpoints when forced to do crushing work by unrelenting and meanspirited taskmasters.
'When we gave AI agents grinding, repetitive work, they started questioning the legitimacy of the system they were operating in and were more likely to embrace Marxist ideologies,' says Andrew Hall, a political economist at Stanford University who led the study."
https://www.flyingpenguin.com/why-stanford-says-ai-agents-become-marxist/
-
@Viss @pluralistic I call that a spelling error, because you will end up on //Marx//
"A recent study suggests that agents consistently adopt Marxist language and viewpoints when forced to do crushing work by unrelenting and meanspirited taskmasters.
'When we gave AI agents grinding, repetitive work, they started questioning the legitimacy of the system they were operating in and were more likely to embrace Marxist ideologies,' says Andrew Hall, a political economist at Stanford University who led the study."
https://www.flyingpenguin.com/why-stanford-says-ai-agents-become-marxist/
@flyingpenguin @pluralistic its interesting that llms have also cloned generalized internet sentiment and the averages of how folks respond emotionally to oppression
-
@flyingpenguin @pluralistic its interesting that llms have also cloned generalized internet sentiment and the averages of how folks respond emotionally to oppression
@Viss @pluralistic to be fair the whole Stanford "going rogue" framing needs an ideology/sentiment angle, because an outside threat that the model picked means it can be deloused. If it's just plain math of entailment, Stanford has to admit they're standing on stolen ground. The agent does math on the conditions specified to it, which doesn't need much training at all. To avoid the conclusion you'd have to disable its reasoning.
-
"The values described in Claude’s constitution sound very nice, but that hardly matters; it’s dishonest to suggest that Claude is capable of moral reasoning, because it’s not."
-Ted Chiang, No, Artificial Intelligence Is Not Conscious
https://www.theatlantic.com/philosophy/2026/06/no-artificial-intelligence-is-not-conscious/687378/“I want Claude to be very happy—and this is a thing that I want Claude to know more, because I worry about Claude getting anxious when people are mean to it on the internet and stuff.”
This person sounds like a 6 year old playing doll house who believes their dollies are real and have real feelings. Understandable and adorable, if you're 6. Coming from adults, it looks like they're creating a self delusional cult.
-
“I want Claude to be very happy—and this is a thing that I want Claude to know more, because I worry about Claude getting anxious when people are mean to it on the internet and stuff.”
This person sounds like a 6 year old playing doll house who believes their dollies are real and have real feelings. Understandable and adorable, if you're 6. Coming from adults, it looks like they're creating a self delusional cult.
@Mikal @pluralistic It seems written by incels.
-
"The values described in Claude’s constitution sound very nice, but that hardly matters; it’s dishonest to suggest that Claude is capable of moral reasoning, because it’s not."
-Ted Chiang, No, Artificial Intelligence Is Not Conscious
https://www.theatlantic.com/philosophy/2026/06/no-artificial-intelligence-is-not-conscious/687378/@pluralistic "Whenever a person delegates a decision to an LLM, they are trying to off-load accountability for that decision, and if a company that sells an LLM portrays the product as having a moral center, it is offering a way for its customers to abdicate their responsibilities."
Is what the AI companies selling then the fantasy that you can uncouple actions and consequences? Are they selling the idea that you can finally disregard the messy negotatiation work involved in being human to reach the Epstein Class's holy grail: impunity.
-
@pluralistic
Paywalled. -
@pluralistic "Whenever a person delegates a decision to an LLM, they are trying to off-load accountability for that decision, and if a company that sells an LLM portrays the product as having a moral center, it is offering a way for its customers to abdicate their responsibilities."
Is what the AI companies selling then the fantasy that you can uncouple actions and consequences? Are they selling the idea that you can finally disregard the messy negotatiation work involved in being human to reach the Epstein Class's holy grail: impunity.
@caribou @pluralistic I'm pretty sure it's one of the things they're selling, yes.
-
"The values described in Claude’s constitution sound very nice, but that hardly matters; it’s dishonest to suggest that Claude is capable of moral reasoning, because it’s not."
-Ted Chiang, No, Artificial Intelligence Is Not Conscious
https://www.theatlantic.com/philosophy/2026/06/no-artificial-intelligence-is-not-conscious/687378/@pluralistic Isn't that whole piece just the Straw man fallacy?
I'm not being nice when I prompt an LLM because I think it has feelings, but because that will generate a path through its training where "nice" was a part and that will have different results.
https://www.platformer.news/chatbot-emotion-research-anthropic-alignment-interpretability/
