Even when you try to use chatGPT like a dry research tool it will preferentially generate text that contains statements to affirm your feelings.
-
Maybe the people at OpenAI understand their contraption well enough that they have a strategy like that.. but I sort of doubt it.
I think it's more a result for them surveying users and selecting models that get good ratings (especially picking models that contain more sycophanic language.)
It's been a while since I touched a ChatGPT model (that I know of).@swope @futurebird Claude builds a model of how you interact with the bot, and pretty definitely changes its replies accordingly. They have more than one script for manipulating you.
-
Everyone already makes fun of LLM sycophancy. "Excellent observation, would you like me to draw up a list of ..."
And when I mention LLM sycophancy someone always mutters "but they have reduced it now"
No they have not. They've made it less obvious. This strange emotional pretend person tugging is a core part of this "product" and its only remarkable feature. Is it enough to justify the gross distortion of the markets? Probably not. But it is ... IDK an innovation in advertising.
@futurebird
I imagine it's a combination of its training data, which allows it to be able to produce this kind of language, and reinforcement learning, which makes it come out at the frequency it does. Plus of course the system prompt also biases to these patterns.Once, I had a coding LLM cycle between abject apology when it failed and very confident optimism that this time it would succeed, over and over again. Completely infuriating - it was hilarious.
I can see that *some* kind of responsiveness to human emotion is needed for the tool not to be completely infuriating though. In the case above that failed, but any written text will be interpreted on an emotional level by humans. Does a "neutral" response exist?
-
Yes, we need to talk about these tricks and strategies much more because they do work on most humans, especially those not forewarned.
@Landa @futurebird event with forewarning sometimes all you can do is reduce exposure, because your brain works how your brain works.
-
@Landa @futurebird event with forewarning sometimes all you can do is reduce exposure, because your brain works how your brain works.
Yes. It’s insidious.
One may consciously know what tricks are applied but other parts of the brain see a „friend-shaped“ thing and insist that something that talks must be a person.Minimizing exposure seems to be the safest choice for most people
-
@futurebird
I skip every ad. Ignore every ad. Answer every survey question "Are you aware of..." with No. "How likely are you to recommend..." ZERO.Done with these games. It's like they don't know my brothers and sisters are in hell and the world is on fire and I'm supposed to play along? Fuckin NO.
Skipping and doing your best to ignore is a good start. But they still influence us in ways we can't anticipate.
I have zero use for a giant pickup truck, I can't afford one, and I still want one. I'm angry at myself for wanting one, and I desperately hope I'm too smart to ever buy one. But no matter how smart I think I am, it's clear that ten thousand commercials with Bob Seger's music and Sam Elliott's voice and absurd fantasy rugged individuals has sunk hooks into me that I can't dislodge.
The real solution is ad-blockers of every possible kind and as much as it's practical (it often isn't), avoiding environments that contain ads.
-
@futurebird @SomeVeganCheeseIsOk This made me think of the inteview where Sigourney Weaver said shooting guns for Aliens made her feel powerful *and she didn't like that* - we can & should always be prepared to interrogate our feelings, even & especially when they're nominally "positive", even empowering.
@jwcph I mean look at Americans. They all drive massive trucks to feel like they are above everyone else and better than them, but in reality, they are powerless serfs who have a pedophile in office.
-
Everyone already makes fun of LLM sycophancy. "Excellent observation, would you like me to draw up a list of ..."
And when I mention LLM sycophancy someone always mutters "but they have reduced it now"
No they have not. They've made it less obvious. This strange emotional pretend person tugging is a core part of this "product" and its only remarkable feature. Is it enough to justify the gross distortion of the markets? Probably not. But it is ... IDK an innovation in advertising.
@futurebird my question for ai boosters. If the ai is so great and all powerful, who is the "them"? Oh, you mean its actually teams of tens of thousands that are modifying results to better fit their agenda? Weird. So its not ai is it. Unless you mean Actual Indians.
-
I've only found one way to deal with advertising in my life: I block them all.
Other people will claim they are immune to propaganda but I don't believe them.
@futurebird "Other people will claim they are immune to propaganda but I don't believe them."
Such people are especially susceptible to propaganda, because they can't see it for the trees.
(Edit: wrong word. Oops. Lol)
-
@futurebird Cancel your subscription and don’t use AI. You know what it is and who that money goes to. You know better. Do better.
I don't pay for it. I want test it one more time in about two months. I has "improved" over the the year. It lists far more source links and makes fewer mistakes.
It still can't really save me much time, and not enough to pay for it at all since everything it spits out needs to be checked.
Were I more excited about it I'd feel the urgency to just block it right now.
-
The slight of hand is to make you forget, or stop caring that "there isn't anyone else" -- the quality of the list of links was better than when I have used the tool in the past, but looking at it again it didn't really find anything I couldn't have found myself in the same time... even with the sad state of most search engines.
But reading "it can be frustrating" did make me feel better about the process. That little trick can't possibly keep working. It seems like a bad idea to lean into it?
@futurebird
Quite different from my reaction to it. I read it as patronising and creepy. -
I'll say one more thing, if the text generator has eased you into talking with it like it's a person. If it's felt easy to use and helpful, or if it has cheered you up with a compliment don't beat yourself up about that too much, but please do notice that it's happening.
It's like commercial with a beautiful person who makes you wish you could be like them. Or a salesman who compliments you. That's all this is.
What is it doing in the coding, research, work tool?
Great question.
@futurebird we don't know *why* it is there, but we do know what it will be used for: supply chain attacks.
-
I've only found one way to deal with advertising in my life: I block them all.
Other people will claim they are immune to propaganda but I don't believe them.
@futurebird Every possible way with every possible tool, I strive to live an ad free life.
Its shocking when I encounter it now. -
@futurebird Every possible way with every possible tool, I strive to live an ad free life.
Its shocking when I encounter it now.Living in a world without ads is a big part of what I think of as luxury if that makes any sense?
Listen, we are never going to get a big feature in The Atlantic or The New Yorker about how healthy, mentally freeing, and spiritually grounding it can be to purge all advertisements from your life, or about how so often this is impossible. (Dear reader can you infer why this will never happen?)
But, maybe we can start a stealth campaign of our own. What is freedom? No ads.
-
Living in a world without ads is a big part of what I think of as luxury if that makes any sense?
Listen, we are never going to get a big feature in The Atlantic or The New Yorker about how healthy, mentally freeing, and spiritually grounding it can be to purge all advertisements from your life, or about how so often this is impossible. (Dear reader can you infer why this will never happen?)
But, maybe we can start a stealth campaign of our own. What is freedom? No ads.
Ad-free spaces are already coded this way culturally. I don't think it'd be that hard.
-
Living in a world without ads is a big part of what I think of as luxury if that makes any sense?
Listen, we are never going to get a big feature in The Atlantic or The New Yorker about how healthy, mentally freeing, and spiritually grounding it can be to purge all advertisements from your life, or about how so often this is impossible. (Dear reader can you infer why this will never happen?)
But, maybe we can start a stealth campaign of our own. What is freedom? No ads.
@futurebird It definitely is luxury; I pay to have ad free services. I can afford it, but many cannot.
-
I think we need to be much more open about discussing these strategies and how well they can work.
It's always a little embarrassing to admit that the text generating machine made me feel anything at all. After-all we all agree I am too brilliant and rational for such a thing to ever happen.
But I was thinking about how that search was "so easy" and when I analyzed WHY it's because of the emotional component not the content of the search.
@futurebird @SomeVeganCheeseIsOk i actually find the extra language makes the search harder - more words to parse to get to the meaning
Just give me the facts, please! -
@futurebird It definitely is luxury; I pay to have ad free services. I can afford it, but many cannot.
@starlily @futurebird Sometimes even when you do pay, they add ads later.
And by sometimes, I mean Usually.
-
Ad-free spaces are already coded this way culturally. I don't think it'd be that hard.
@futurebird @starlily I think we should associate ads with harassment. Especially personalized ones.
-
@futurebird @SomeVeganCheeseIsOk i actually find the extra language makes the search harder - more words to parse to get to the meaning
Just give me the facts, please!@leadegroot @futurebird I am spoiled by searches that provide niche blogs on unpopular topics
-
@starlily @futurebird Sometimes even when you do pay, they add ads later.
And by sometimes, I mean Usually.
@Epic_Null @futurebird Not wrong. I watch enough YT content that paying for it to remove ads is worthwhile, but any creator of any size is now forced to have "in-content promotion" that of course isnt blocked in any way other than the fast forward button.
