I've never been opposed to the word "hallucinating" for describing how AI makes mistakes ... until now.
-
I've never been opposed to the word "hallucinating" for describing how AI makes mistakes ... until now.
I just talked to someone who thought AI hallucinations would be obvious because it would be obvious if you talked to a *person* who was hallucinating.
In other words, they equated "hallucination" with "sounds wacko" and accepted AI output as true because it sounded level headed.
1/2
@grammargirl You might get traction by describing it as "truthy" (and explaining that), or by noting that you'll get basically the same results by asking "What would a response to the question '(original question)' sound like?"
Note that "what it would sound like" is very much not the same as "what is the answer" - but what you get will sure *sound like* an answer.
-
@grammargirl Hm. It’s not always obvious if a person you are talking to is hallucinating, depending on what their hallucinations are and what they say.
I get their point, just am sad on the mental illness rep side.
@grammargirl@zirk.us @queenofnewyork@newsie.social
I was thinking this too, if someone thinks they can tell if a person is hallucinating because it's "obvious", then they have a major misunderstanding of how hallucinations work at all.
I get where the concern is for AI hallucinations as a term, but then the same concern is there for hallucinations a human person has too.
-
I've never been opposed to the word "hallucinating" for describing how AI makes mistakes ... until now.
I just talked to someone who thought AI hallucinations would be obvious because it would be obvious if you talked to a *person* who was hallucinating.
In other words, they equated "hallucination" with "sounds wacko" and accepted AI output as true because it sounded level headed.
1/2
all the outputs of LLMs and the like are hallucinations, it's just that the "bell curve" of the outputs overlap the appearance of most of what the user wants
-
I've never been opposed to the word "hallucinating" for describing how AI makes mistakes ... until now.
I just talked to someone who thought AI hallucinations would be obvious because it would be obvious if you talked to a *person* who was hallucinating.
In other words, they equated "hallucination" with "sounds wacko" and accepted AI output as true because it sounded level headed.
1/2
They are AI mirages: they look like what you asked for but the closer you look the less there is.
Only users can hallucinate.
-
I've never been opposed to the word "hallucinating" for describing how AI makes mistakes ... until now.
I just talked to someone who thought AI hallucinations would be obvious because it would be obvious if you talked to a *person* who was hallucinating.
In other words, they equated "hallucination" with "sounds wacko" and accepted AI output as true because it sounded level headed.
1/2
@grammargirl i had a discussion with someone who thought the screen would go fuzzy or similar when AI was hallucinating. So they thought it would be obvious
-
I'm iffy on the term. But I don't have anything better.
But this: GenAI doesn't sometimes hallucinate. It always hallucinates. It only ever hallucinates.
Sometimes, what it hallucinates is plausible.
@draNgNon @grammargirl@BenAveling @draNgNon @grammargirl
The AI is generating language from some matrix algebra that regurgitates transforms of the test data or mirages of it. Only users can hallucinate and believe the mirages are real while a whirring vortex of vectors can't believe in anything.
-
The word "hallucination" isn't going away — it's a widely used industry term — but we need to explain it better for beginners:
"Hallucination" is just a fancy word for "confidently makes mistakes":
"Remember: AI hallucinates, and you need to confirm all facts" should be something like "Remember: AI confidently makes mistakes, and you need to confirm all facts" or "AI tells you things that are wrong in a way that sounds completely believable. Confirm all facts!"
@grammargirl I think it's funny that people who object to the use of 'halluctinate' because it anthropomorphises AI are nonetheless happy with their use of the word 'confident', as in 'confidently makes mistakes', in the same context.
-
@grammargirl This is a good example of why that term is so dangerous. Thank you for posting it.
That said, while I have zero hope of making that term go away, we also have the word "slop" as a counter.
"Ugh. It had a hallucination..."
"Yup. And the results are now slop."
That said, I don't myself use "hallucination" in the "AI" context. I refer to the error rate, which last I checked, hovered around 40%.
@orionkidder @grammargirl I’ve heard the Spanish science communicator Ignacio Crespo argue that “hallucination” is misleading in this context, because it imports a human mental-state metaphor into a statistical text-generation error. “Confabulation” may be closer: a plausible-sounding reconstruction that fills gaps. Still, it also comes from human cognition, so it can anthropomorphise the model too.
-
@orionkidder @grammargirl I’ve heard the Spanish science communicator Ignacio Crespo argue that “hallucination” is misleading in this context, because it imports a human mental-state metaphor into a statistical text-generation error. “Confabulation” may be closer: a plausible-sounding reconstruction that fills gaps. Still, it also comes from human cognition, so it can anthropomorphise the model too.
@orionkidder @grammargirl I think the deeper problem with “hallucination” is that it imports a human mental-state metaphor into a statistical text-generation error. That can make people expect obviously bizarre output, when the real danger is often confident, plausible-sounding falsehoods. “Confabulation” has a similar problem, though. But, I don’t know, it sounds better to me.
-
@AccordionBruce @orionkidder @grammargirl
Exactly this.
Hallucination is an act of cognition. The machine doesn't@RnDanger @AccordionBruce @orionkidder @grammargirl
It seems such a pointless, minor nuance that will make no difference whatsoever in practice
(yes I am aware talking about this kind of minor nuances is your day job, but still, someone's gotta say it)
-
I've never been opposed to the word "hallucinating" for describing how AI makes mistakes ... until now.
I just talked to someone who thought AI hallucinations would be obvious because it would be obvious if you talked to a *person* who was hallucinating.
In other words, they equated "hallucination" with "sounds wacko" and accepted AI output as true because it sounded level headed.
1/2
@grammargirl
Some of the kookiest genuinely bat nuts crazy people Ive ever met, spoke exceptionally well, and logically connected ideas together. They could make exceptionally convincing arguments that were nonetheless wrong."Spoke eloquently" is a lower bar than some assume.
-
The word "hallucination" isn't going away — it's a widely used industry term — but we need to explain it better for beginners:
"Hallucination" is just a fancy word for "confidently makes mistakes":
"Remember: AI hallucinates, and you need to confirm all facts" should be something like "Remember: AI confidently makes mistakes, and you need to confirm all facts" or "AI tells you things that are wrong in a way that sounds completely believable. Confirm all facts!"
@grammargirl
I'm opposed to your use of 'AI'. An LLM is not an intelligence, even though that is what people call it.
Every word the industry likes for its own products probably helps to mislead the public.
Every form of anthropomorphisation of LLMs should be banned. -
@RnDanger @AccordionBruce @orionkidder @grammargirl
It seems such a pointless, minor nuance that will make no difference whatsoever in practice
(yes I am aware talking about this kind of minor nuances is your day job, but still, someone's gotta say it)
@gotofritz @RnDanger @AccordionBruce @orionkidder @grammargirl
Language can be used as one of the most dangerous tools we have because it shapes the way we think (and thus our future) mostly on a subconscious level. The more subtly a word misleads, the more difference it can make in practice. -
I've never been opposed to the word "hallucinating" for describing how AI makes mistakes ... until now.
I just talked to someone who thought AI hallucinations would be obvious because it would be obvious if you talked to a *person* who was hallucinating.
In other words, they equated "hallucination" with "sounds wacko" and accepted AI output as true because it sounded level headed.
1/2
@grammargirl This is a great Wittgenstein conundrum but to be honest I would leave it as is, the scientific community will find its own terms in publications; we are otherwise living dangerous times and the last thing we want is to split hairs and divert people from the very issue at hand. Personally I am decanting for a good Côtes Du Rone for my hallucinations, and of course some squirt of AI.
-
@orionkidder @grammargirl I think the deeper problem with “hallucination” is that it imports a human mental-state metaphor into a statistical text-generation error. That can make people expect obviously bizarre output, when the real danger is often confident, plausible-sounding falsehoods. “Confabulation” has a similar problem, though. But, I don’t know, it sounds better to me.
@danielmunoz @grammargirl This is why I refer to its "error rate." It's a machine that produces false answers to such a large degree that it shouldn't be trusted. It's simply faulty.
-
@gotofritz @RnDanger @AccordionBruce @orionkidder @grammargirl
Language can be used as one of the most dangerous tools we have because it shapes the way we think (and thus our future) mostly on a subconscious level. The more subtly a word misleads, the more difference it can make in practice.@elfburgerman @gotofritz @RnDanger @AccordionBruce @grammargirl I think this is true. Like I said above, I have zero expectation that my language use is going to make a damn bit of difference at scale, but in individual conversations, refusing the metaphor of consciousness can help reframe.
It's just an error. The machine is faulty. It makes errors a lot.
-
@gotofritz @RnDanger @AccordionBruce @orionkidder @grammargirl
Language can be used as one of the most dangerous tools we have because it shapes the way we think (and thus our future) mostly on a subconscious level. The more subtly a word misleads, the more difference it can make in practice.@elfburgerman @gotofritz @AccordionBruce @orionkidder @grammargirl
I agree.
"Hallucination" is a great marketing term to make people want to trust a machine, but it's a pretty poor choice of words to convey any understanding of what the machine does or how it does it -
The word "hallucination" isn't going away — it's a widely used industry term — but we need to explain it better for beginners:
"Hallucination" is just a fancy word for "confidently makes mistakes":
"Remember: AI hallucinates, and you need to confirm all facts" should be something like "Remember: AI confidently makes mistakes, and you need to confirm all facts" or "AI tells you things that are wrong in a way that sounds completely believable. Confirm all facts!"
@grammargirl
> The word "hallucination" ... it's a widely used industry termIt is a widely used industry lie that regurgirators do not lie but somehow are slightly mistaken.
While technically it is a "less expected but still possible words rehashing output" or "imperfect probability glitch" or like, the "lie" term has the accurate and precise definiens of what the output is factually. So that term should be used. I hope it soon will be obligatory for "the industry" to use in the EU.
:))
-
@elfburgerman @gotofritz @AccordionBruce @orionkidder @grammargirl
I agree.
"Hallucination" is a great marketing term to make people want to trust a machine, but it's a pretty poor choice of words to convey any understanding of what the machine does or how it does it@RnDanger @elfburgerman @gotofritz @AccordionBruce @grammargirl Exactly. Making machines seem like magic, seem like they have no internal mechanism, is a common tactic. It's why we refer to external hard drives that we don't own or control as "the cloud."
-
The word "hallucination" isn't going away — it's a widely used industry term — but we need to explain it better for beginners:
"Hallucination" is just a fancy word for "confidently makes mistakes":
"Remember: AI hallucinates, and you need to confirm all facts" should be something like "Remember: AI confidently makes mistakes, and you need to confirm all facts" or "AI tells you things that are wrong in a way that sounds completely believable. Confirm all facts!"
@grammargirl Definitely the latter, but with a slight addition:
"AI tells you things that are wrong in a way that sounds completely believable - which is the system functioning as designed. Confirm all facts!"
