Even when you try to use chatGPT like a dry research tool it will preferentially generate text that contains statements to affirm your feelings.
-
Even when you try to use chatGPT like a dry research tool it will preferentially generate text that contains statements to affirm your feelings.
My interaction with it is minimal. I have a year long premium subscription because of a work-sponsored workshop. Rather than totally ignore it I ask it the same kind of questions every few months to see how it works. I first asked it for "good papers about the arboreal ant nests called 'ant gardens'?"
It gave me a numbered list with links.
But then:
@futurebird The groveling, obsequious responses make me cringe.
-
@futurebird persuasion should be taught in schools so folks know how it works on them.
@SomeVeganCheeseIsOk @futurebird I took a college class on essentially that premise called "Practical Reasoning". We studied from the book "Influence" by Robert Cialdini.
It really should be a standard socioeconomic self defense class.
-
@futurebird persuasion should be taught in schools so folks know how it works on them.
I think we need to be much more open about discussing these strategies and how well they can work.
It's always a little embarrassing to admit that the text generating machine made me feel anything at all. After-all we all agree I am too brilliant and rational for such a thing to ever happen.
But I was thinking about how that search was "so easy" and when I analyzed WHY it's because of the emotional component not the content of the search.
-
@futurebird The groveling, obsequious responses make me cringe.
They make us cringe when we notice them. But if you use these tools regularly you will stop noticing them.
-
Do, you notice it?
"can be frustrating"
I didn't say I was "frustrated" I just asked for more photos because most of the papers in the first response didn't have any.
HOWEVER, the guess that I was frustrated was correct. It even felt nice to read that some one else was having the same ... except wait.
There isn't anyone else!
I thought I'd share this to highlight just how deeply embedded these techniques are. Do you dislike being manipulated?
Please notice this.
@futurebird I keep hearing people refer to it as "he" and it really bugs me, such a dangerous mental framing
-
@futurebird I keep hearing people refer to it as "he" and it really bugs me, such a dangerous mental framing
@paulcox @futurebird it would 've been cool if jesus and all of those people were an ant
-
@futurebird I keep hearing people refer to it as "he" and it really bugs me, such a dangerous mental framing
@paulcox @futurebird it seems SO dangerous to me that so many people are having a chatbot tell them how they feel all the time. Like properly, long-lastingly damaging
-
Do, you notice it?
"can be frustrating"
I didn't say I was "frustrated" I just asked for more photos because most of the papers in the first response didn't have any.
HOWEVER, the guess that I was frustrated was correct. It even felt nice to read that some one else was having the same ... except wait.
There isn't anyone else!
I thought I'd share this to highlight just how deeply embedded these techniques are. Do you dislike being manipulated?
Please notice this.
@futurebird Do I like being manipulated? Probably! It's amazing to feel like I'm a special boy, that I'm clever and exceptional...
It's the *having been* manipulated that really stinks though, not worth it at all. Really sours me to the concept tbh.
-
I think we need to be much more open about discussing these strategies and how well they can work.
It's always a little embarrassing to admit that the text generating machine made me feel anything at all. After-all we all agree I am too brilliant and rational for such a thing to ever happen.
But I was thinking about how that search was "so easy" and when I analyzed WHY it's because of the emotional component not the content of the search.
@futurebird @SomeVeganCheeseIsOk
I am once again reminded of the LLMentalist article explaining how LLM proponents are cold-reading themselves, like they are working for the circus.
-
@futurebird persuasion should be taught in schools so folks know how it works on them.
@SomeVeganCheeseIsOk @futurebird instead we're taught "free will" guarantees we can't be persuaded unless we actually want it.
-
The slight of hand is to make you forget, or stop caring that "there isn't anyone else" -- the quality of the list of links was better than when I have used the tool in the past, but looking at it again it didn't really find anything I couldn't have found myself in the same time... even with the sad state of most search engines.
But reading "it can be frustrating" did make me feel better about the process. That little trick can't possibly keep working. It seems like a bad idea to lean into it?
@futurebird on the occasions I have to interact with chatbots, this is what I find uncanny-valley creepy. When they express empathy or try to make friends, it feels like a trap. And yet, many people obviously like that kind of interaction.
-
The slight of hand is to make you forget, or stop caring that "there isn't anyone else" -- the quality of the list of links was better than when I have used the tool in the past, but looking at it again it didn't really find anything I couldn't have found myself in the same time... even with the sad state of most search engines.
But reading "it can be frustrating" did make me feel better about the process. That little trick can't possibly keep working. It seems like a bad idea to lean into it?
Maybe the people at OpenAI understand their contraption well enough that they have a strategy like that.. but I sort of doubt it.
I think it's more a result for them surveying users and selecting models that get good ratings (especially picking models that contain more sycophanic language.)
It's been a while since I touched a ChatGPT model (that I know of). -
Do, you notice it?
"can be frustrating"
I didn't say I was "frustrated" I just asked for more photos because most of the papers in the first response didn't have any.
HOWEVER, the guess that I was frustrated was correct. It even felt nice to read that some one else was having the same ... except wait.
There isn't anyone else!
I thought I'd share this to highlight just how deeply embedded these techniques are. Do you dislike being manipulated?
Please notice this.
@futurebird There is something I have noticed about the web chat interface for GPT (and Claude) that I cannot now unsee. When I send a message, the text area slides up to hide the previous messages, and the reply then reveals itself over the next few seconds.
This engages me, I have no doubt; and not in a way I can think myself out of. Why? Because it's a slot machine! You pull the lever, it spins, and you get your "prize". It's not a neutral interface.
-
J jwcph@helvede.net shared this topic
-
I think we need to be much more open about discussing these strategies and how well they can work.
It's always a little embarrassing to admit that the text generating machine made me feel anything at all. After-all we all agree I am too brilliant and rational for such a thing to ever happen.
But I was thinking about how that search was "so easy" and when I analyzed WHY it's because of the emotional component not the content of the search.
@futurebird @SomeVeganCheeseIsOk This made me think of the inteview where Sigourney Weaver said shooting guns for Aliens made her feel powerful *and she didn't like that* - we can & should always be prepared to interrogate our feelings, even & especially when they're nominally "positive", even empowering.
