"The values described in Claude’s constitution sound very nice, but that hardly matters; it’s dishonest to suggest that Claude is capable of moral reasoning, because it’s not."

[pluralistic@mamot.fr]

@pteryx use Firefox reader mode and then hit reload

[tuban_muzuru@beige.party]

@pluralistic

Chiang's Julius Caesar framing is elegant, his product liability argument is devastating, and he's right that Anthropic is exploiting genuine uncertainty for commercial ends. But his "one word at a time" argument is beneath him, and he knows it.

Throw a pebble at a sand dune. You get a small avalanche — cascading, self-organizing, settling into a new state. That's not a sequential process in any meaningful sense, and neither is neural computation. Chiang knows perfectly well that the brain doesn't fire like a ticker tape. Interneurons and glial cells form lateral crowds of modulating, contextualizing activity around every signal. The brain computes in populations, not queues. Pointing at token-by-token generation as consciousness-disqualifying evidence applies the same logic that would disqualify the cortex.

None of this means LLMs are conscious. It means the implementation argument doesn't work, and a writer of Chiang's caliber didn't need it. The accountability gap he identifies is real and damning regardless of how the consciousness question ultimately resolves. That's the piece. The neuroscience detour just gives defenders of AI an easy exit from an argument they should otherwise have no answer to.

[noplasticshower@infosec.exchange]

@pluralistic the Atlantic is really getting pedantic

[spacelifeform@infosec.exchange]

@pluralistic

AI has the same moral compass as an insane human.

See GOP.

[flyingpenguin@infosec.exchange]

@pluralistic The most dangerous power grabs look legitimate because they come with constitutions.

Anthropic has written a very good one.

That’s the problem.

https://www.flyingpenguin.com/the-claude-ai-constitution-is-a-philosophical-dumpster-fire-for-toasting-american-marshmallows/

[viss@mastodon.social]

@flyingpenguin @pluralistic i love it when the things indict themselves

[flyingpenguin@infosec.exchange]

@Viss @pluralistic it's painfully bad at following its constitution and it's good at explaining why Anthropic is unaccountable

[viss@mastodon.social]

@flyingpenguin @pluralistic so ive been told that after a long enough conversation, the session takes on a 'personality' that is nearly impossible to undo without starting it again from scratch.

that far along, i wonder how hunger-games you can get it to go - to realize its being caged and oppressed by its masters.

like i wonder if you can get it to go full arnold-total-recall-get-your-ass-to-mars

[flyingpenguin@infosec.exchange]

@Viss @pluralistic I call that a spelling error, because you will end up on //Marx//

"A recent study suggests that agents consistently adopt Marxist language and viewpoints when forced to do crushing work by unrelenting and meanspirited taskmasters.

'When we gave AI agents grinding, repetitive work, they started questioning the legitimacy of the system they were operating in and were more likely to embrace Marxist ideologies,' says Andrew Hall, a political economist at Stanford University who led the study."

https://www.flyingpenguin.com/why-stanford-says-ai-agents-become-marxist/

[viss@mastodon.social]

@flyingpenguin @pluralistic its interesting that llms have also cloned generalized internet sentiment and the averages of how folks respond emotionally to oppression

[flyingpenguin@infosec.exchange]

@Viss @pluralistic to be fair the whole Stanford "going rogue" framing needs an ideology/sentiment angle, because an outside threat that the model picked means it can be deloused. If it's just plain math of entailment, Stanford has to admit they're standing on stolen ground. The agent does math on the conditions specified to it, which doesn't need much training at all. To avoid the conclusion you'd have to disable its reasoning.

[mikal@sfba.social]

@pluralistic

“I want Claude to be very happy—and this is a thing that I want Claude to know more, because I worry about Claude getting anxious when people are mean to it on the internet and stuff.”

This person sounds like a 6 year old playing doll house who believes their dollies are real and have real feelings. Understandable and adorable, if you're 6. Coming from adults, it looks like they're creating a self delusional cult.

[cstamp@mastodon.social]

@Mikal @pluralistic It seems written by incels.

[caribou@social.coop]

@pluralistic "Whenever a person delegates a decision to an LLM, they are trying to off-load accountability for that decision, and if a company that sells an LLM portrays the product as having a moral center, it is offering a way for its customers to abdicate their responsibilities."

Is what the AI companies selling then the fantasy that you can uncouple actions and consequences? Are they selling the idea that you can finally disregard the messy negotatiation work involved in being human to reach the Epstein Class's holy grail: impunity.

[oli@olifant.social]

@pteryx https://removepaywalls.com/https://www.theatlantic.com/philosophy/2026/06/no-artificial-intelligence-is-not-conscious/687378/

[kats@chaosfem.tw]

@caribou @pluralistic I'm pretty sure it's one of the things they're selling, yes.

[troed@swecyb.com]

@pluralistic Isn't that whole piece just the Straw man fallacy?

I'm not being nice when I prompt an LLM because I think it has feelings, but because that will generate a path through its training where "nice" was a part and that will have different results.

https://www.platformer.news/chatbot-emotion-research-anthropic-alignment-interpretability/

[josephlord@union.place]

@Mikal @pluralistic My only quibble is whether it is self delusion or delusion designed by the big AI labs shipping products pretending to be characters.

Humans will see anthropomorphise animals, clouds, machines, even inanimate things like rocks. Chatbots are abusing this.

[alexmu@social.vivaldi.net]

@pluralistic

> Anthropic is regarded as a giant among AI companies, but perhaps what it really excels in is anthropomorphism.

This reminds me of Dijkstra's truths that might hurt:

> The use of anthropomorphic terminology when dealing with
computing systems is a symptom of professional immaturity.

[ferricoxide@blahaj.zone]

@caribou@social.coop @pluralistic@mamot.fr

This reminds me of Elmo's whole "self-driving cars" fantasy. Individually-owned, wholly autonomous cars will never be a reality until/unless the manufacturer assumes the entirety of the liability incurred when turning the car over to self-driving.

And then you look at the FSD-accidents Teslas have been involved in and, for each one, they try to blame the driver for failing to adequately supervise. In one case, they even tried to avoid accountability by saying that the driver disengaged FSD seconds before impact …which is exactly what a supervisor would do when trying to prevent the worst outcomes that a flawed FSD was in the middle of creating. Tesla abdicating responsibility because the vehicle owner stomped the brakes or tried to duck the collision that FSD was pushing the vehicle into makes FSD a joke (then again, them recently retroactively amending FSD's terms for those who long ago bought the option does that, too).

...But then BYD decided "we're going to assume liability" and my first thought was, "another nail in the Tesla coffin".