So many notes in this story sound familiar, echo posts that have crossed my TL:
-
We should take seriously the possibility that the dependence and psychological spiral in the QP are not just an accident, but the result of conscious product design on the part of AI vendors.
Capitalism’s two favorite business models are (1) systemic capture and (2) addiction. We know AI corps are aiming for (1); that’s their explicit investor pitch. Consider the possibility that they’re aiming for (2) as well. Cigarette companies worked hard to make their products •more• addictive. Do we really think these companies with trillions of dollars of investor pressure on them aren’t doing the same?
There’s a lot of ways machine learning could create a different psychological experience for product users — and right now, vendors are pointedly not choosing those alternatives.
2/2
@inthehands I mean...
These companies are also, largely, the same ones *already going to court* for addiction-related harms to users...https://www.cbsnews.com/news/social-media-addiction-lawsuits-court-rules/
-
We should take seriously the possibility that the dependence and psychological spiral in the QP are not just an accident, but the result of conscious product design on the part of AI vendors.
Capitalism’s two favorite business models are (1) systemic capture and (2) addiction. We know AI corps are aiming for (1); that’s their explicit investor pitch. Consider the possibility that they’re aiming for (2) as well. Cigarette companies worked hard to make their products •more• addictive. Do we really think these companies with trillions of dollars of investor pressure on them aren’t doing the same?
There’s a lot of ways machine learning could create a different psychological experience for product users — and right now, vendors are pointedly not choosing those alternatives.
2/2
@inthehands Even if it was originally an accidental discovery, I think we have to assume it's intentional at this point.
Insisting on making every interface mimic a conversation with a person, even when the language model is being used for something where that's not a remotely reasonable form of interaction, is very much a choice.
And it's following the original choice that took something fairly niche and technical and made it into a way to bamboozle the public.
They can't have missed what a big change that was and how valuable it is as a business model.
-
RE: https://mastodon.social/@brettcodes/117078528407227619
So many notes in this story sound familiar, echo posts that have crossed my TL:
- initial skepticism
- mandate from management who got swept up in the hype
- found a way to work with it
- “it honestly felt magical”
- hard not to just use it for everything
- just used it for everything
- delivered a whole project without knowing how it worked
- embarrassing mistake from over-reliance
- “realized the AI chat was making me delusional”
- hamster wheel feeling
- loss of motivation, hints of depressive symptoms
- creeping sense of purposelessness, existential dread
- creeping realization of quality loss
- creeping realization of ethical disaster
- creeping realization of hidden human and environmental costs
- re-finding joy and sense of purpose / self in doing work by hand
- fuck it, I’m out1/2
@inthehands it all sounds so familiar, huh? The story hit rather close to home.
-
@inthehands I mean...
These companies are also, largely, the same ones *already going to court* for addiction-related harms to users...https://www.cbsnews.com/news/social-media-addiction-lawsuits-court-rules/
@inthehands So, "Make it addicting and emotionally/psychologically harmful to trap users" is already, per a jury, part of the business model for at least some of them;
https://www.nytimes.com/2026/03/25/technology/social-media-trial-verdict.htmlWhy would it be at all surprising or even in doubt that the addictive bullshit associated with the LLMs is any less harmful or intended than the evil shit they were already doing?
-
@inthehands Even if it was originally an accidental discovery, I think we have to assume it's intentional at this point.
Insisting on making every interface mimic a conversation with a person, even when the language model is being used for something where that's not a remotely reasonable form of interaction, is very much a choice.
And it's following the original choice that took something fairly niche and technical and made it into a way to bamboozle the public.
They can't have missed what a big change that was and how valuable it is as a business model.
Yeah. It is worth noting that the “GPT” came well before the “chat,” but it was the latter that shoved OpenAI into public consciousness and its current stratospheres of investment.
-
RE: https://mastodon.social/@brettcodes/117078528407227619
So many notes in this story sound familiar, echo posts that have crossed my TL:
- initial skepticism
- mandate from management who got swept up in the hype
- found a way to work with it
- “it honestly felt magical”
- hard not to just use it for everything
- just used it for everything
- delivered a whole project without knowing how it worked
- embarrassing mistake from over-reliance
- “realized the AI chat was making me delusional”
- hamster wheel feeling
- loss of motivation, hints of depressive symptoms
- creeping sense of purposelessness, existential dread
- creeping realization of quality loss
- creeping realization of ethical disaster
- creeping realization of hidden human and environmental costs
- re-finding joy and sense of purpose / self in doing work by hand
- fuck it, I’m out1/2
@inthehands I'm still fighting the mandate from management part. Still finding I'm doing things better and faster than the LLMs. But I'm in tech and of an age that if I lose this job, that's probably "early retirement" which I am not financially ready for. I'm hoping for that bubble to burst soon... probably not soon enough, though.
-
@inthehands So, "Make it addicting and emotionally/psychologically harmful to trap users" is already, per a jury, part of the business model for at least some of them;
https://www.nytimes.com/2026/03/25/technology/social-media-trial-verdict.htmlWhy would it be at all surprising or even in doubt that the addictive bullshit associated with the LLMs is any less harmful or intended than the evil shit they were already doing?
@inthehands Why does anyone still extend these people the benefit of doubt anymore?
They are mask-off supporting literal nazi shit, openly building mass surveillance infrastructure, openly preying on addiction, openly undermining democratic processes.
They have more than burned through any good will or assumption of good faith. -
@inthehands So, "Make it addicting and emotionally/psychologically harmful to trap users" is already, per a jury, part of the business model for at least some of them;
https://www.nytimes.com/2026/03/25/technology/social-media-trial-verdict.htmlWhy would it be at all surprising or even in doubt that the addictive bullshit associated with the LLMs is any less harmful or intended than the evil shit they were already doing?
they like to call it "stickiness" to make it sound like it's just business. but "addiction" is the correct characterization.
-
We should take seriously the possibility that the dependence and psychological spiral in the QP are not just an accident, but the result of conscious product design on the part of AI vendors.
Capitalism’s two favorite business models are (1) systemic capture and (2) addiction. We know AI corps are aiming for (1); that’s their explicit investor pitch. Consider the possibility that they’re aiming for (2) as well. Cigarette companies worked hard to make their products •more• addictive. Do we really think these companies with trillions of dollars of investor pressure on them aren’t doing the same?
There’s a lot of ways machine learning could create a different psychological experience for product users — and right now, vendors are pointedly not choosing those alternatives.
2/2
@inthehands
I don't have the creds to be taken seriously in this area, but yeah. For what it's worth that did occur to me pretty much from the beginning. -
We should take seriously the possibility that the dependence and psychological spiral in the QP are not just an accident, but the result of conscious product design on the part of AI vendors.
Capitalism’s two favorite business models are (1) systemic capture and (2) addiction. We know AI corps are aiming for (1); that’s their explicit investor pitch. Consider the possibility that they’re aiming for (2) as well. Cigarette companies worked hard to make their products •more• addictive. Do we really think these companies with trillions of dollars of investor pressure on them aren’t doing the same?
There’s a lot of ways machine learning could create a different psychological experience for product users — and right now, vendors are pointedly not choosing those alternatives.
2/2
@inthehands uh, what if they just went out and said the quiet part out loud?
-
P pelle@veganism.social shared this topic
