I've never been opposed to the word "hallucinating" for describing how AI makes mistakes ... until now.
-
I've never been opposed to the word "hallucinating" for describing how AI makes mistakes ... until now.
I just talked to someone who thought AI hallucinations would be obvious because it would be obvious if you talked to a *person* who was hallucinating.
In other words, they equated "hallucination" with "sounds wacko" and accepted AI output as true because it sounded level headed.
1/2
@grammargirl This is a great Wittgenstein conundrum but to be honest I would leave it as is, the scientific community will find its own terms in publications; we are otherwise living dangerous times and the last thing we want is to split hairs and divert people from the very issue at hand. Personally I am decanting for a good Côtes Du Rone for my hallucinations, and of course some squirt of AI.
-
@orionkidder @grammargirl I think the deeper problem with “hallucination” is that it imports a human mental-state metaphor into a statistical text-generation error. That can make people expect obviously bizarre output, when the real danger is often confident, plausible-sounding falsehoods. “Confabulation” has a similar problem, though. But, I don’t know, it sounds better to me.
@danielmunoz @grammargirl This is why I refer to its "error rate." It's a machine that produces false answers to such a large degree that it shouldn't be trusted. It's simply faulty.
-
@gotofritz @RnDanger @AccordionBruce @orionkidder @grammargirl
Language can be used as one of the most dangerous tools we have because it shapes the way we think (and thus our future) mostly on a subconscious level. The more subtly a word misleads, the more difference it can make in practice.@elfburgerman @gotofritz @RnDanger @AccordionBruce @grammargirl I think this is true. Like I said above, I have zero expectation that my language use is going to make a damn bit of difference at scale, but in individual conversations, refusing the metaphor of consciousness can help reframe.
It's just an error. The machine is faulty. It makes errors a lot.
-
@gotofritz @RnDanger @AccordionBruce @orionkidder @grammargirl
Language can be used as one of the most dangerous tools we have because it shapes the way we think (and thus our future) mostly on a subconscious level. The more subtly a word misleads, the more difference it can make in practice.@elfburgerman @gotofritz @AccordionBruce @orionkidder @grammargirl
I agree.
"Hallucination" is a great marketing term to make people want to trust a machine, but it's a pretty poor choice of words to convey any understanding of what the machine does or how it does it -
The word "hallucination" isn't going away — it's a widely used industry term — but we need to explain it better for beginners:
"Hallucination" is just a fancy word for "confidently makes mistakes":
"Remember: AI hallucinates, and you need to confirm all facts" should be something like "Remember: AI confidently makes mistakes, and you need to confirm all facts" or "AI tells you things that are wrong in a way that sounds completely believable. Confirm all facts!"
@grammargirl
> The word "hallucination" ... it's a widely used industry termIt is a widely used industry lie that regurgirators do not lie but somehow are slightly mistaken.
While technically it is a "less expected but still possible words rehashing output" or "imperfect probability glitch" or like, the "lie" term has the accurate and precise definiens of what the output is factually. So that term should be used. I hope it soon will be obligatory for "the industry" to use in the EU.
:))
-
@elfburgerman @gotofritz @AccordionBruce @orionkidder @grammargirl
I agree.
"Hallucination" is a great marketing term to make people want to trust a machine, but it's a pretty poor choice of words to convey any understanding of what the machine does or how it does it@RnDanger @elfburgerman @gotofritz @AccordionBruce @grammargirl Exactly. Making machines seem like magic, seem like they have no internal mechanism, is a common tactic. It's why we refer to external hard drives that we don't own or control as "the cloud."
-
The word "hallucination" isn't going away — it's a widely used industry term — but we need to explain it better for beginners:
"Hallucination" is just a fancy word for "confidently makes mistakes":
"Remember: AI hallucinates, and you need to confirm all facts" should be something like "Remember: AI confidently makes mistakes, and you need to confirm all facts" or "AI tells you things that are wrong in a way that sounds completely believable. Confirm all facts!"
@grammargirl Definitely the latter, but with a slight addition:
"AI tells you things that are wrong in a way that sounds completely believable - which is the system functioning as designed. Confirm all facts!"
-
The word "hallucination" isn't going away — it's a widely used industry term — but we need to explain it better for beginners:
"Hallucination" is just a fancy word for "confidently makes mistakes":
"Remember: AI hallucinates, and you need to confirm all facts" should be something like "Remember: AI confidently makes mistakes, and you need to confirm all facts" or "AI tells you things that are wrong in a way that sounds completely believable. Confirm all facts!"
@grammargirl
I think of this as the nines imbalance.In a datacenter there is talk of nines of uptime. Going from two nines (99%) uptime to three minutes (99.9%) requires an order of magnitude investment. Another again for four nines (99.99%).
The AI nines imbalance is that
It is one nine accurate (90%)
but four nines eloquent (99.99%) -
The word "hallucination" isn't going away — it's a widely used industry term — but we need to explain it better for beginners:
"Hallucination" is just a fancy word for "confidently makes mistakes":
"Remember: AI hallucinates, and you need to confirm all facts" should be something like "Remember: AI confidently makes mistakes, and you need to confirm all facts" or "AI tells you things that are wrong in a way that sounds completely believable. Confirm all facts!"
@grammargirl I appreciate "bullshit" as a better term per this article: https://www.psypost.org/scholars-ai-isnt-hallucinating-its-bullshitting/
-
The word "hallucination" isn't going away — it's a widely used industry term — but we need to explain it better for beginners:
"Hallucination" is just a fancy word for "confidently makes mistakes":
"Remember: AI hallucinates, and you need to confirm all facts" should be something like "Remember: AI confidently makes mistakes, and you need to confirm all facts" or "AI tells you things that are wrong in a way that sounds completely believable. Confirm all facts!"
@grammargirl
Everything LLM based "AI" generates is hallucination. It's just that in more than 50% of cases those hallucinations resemble facts. -
The word "hallucination" isn't going away — it's a widely used industry term — but we need to explain it better for beginners:
"Hallucination" is just a fancy word for "confidently makes mistakes":
"Remember: AI hallucinates, and you need to confirm all facts" should be something like "Remember: AI confidently makes mistakes, and you need to confirm all facts" or "AI tells you things that are wrong in a way that sounds completely believable. Confirm all facts!"
I'd like to suggest that the core of the problem here is that the pace of technological development is outstripping the pace of the evolution of our language to adequately describe it. I think we will have to come up with new words, or at least appropriate some more obscure ones to the cause with updated definitions.
(All that said, I think "botfarts" or, in less polite company, "botshit" kind of works, no?)
-
The word "hallucination" isn't going away — it's a widely used industry term — but we need to explain it better for beginners:
"Hallucination" is just a fancy word for "confidently makes mistakes":
"Remember: AI hallucinates, and you need to confirm all facts" should be something like "Remember: AI confidently makes mistakes, and you need to confirm all facts" or "AI tells you things that are wrong in a way that sounds completely believable. Confirm all facts!"
@grammargirl a lot of people used to say "no it's not *hallucination* it's *confabulation*." Confabulation is the thing the human brain does that is somewhat analogous to what AI does: confidently believing in something that we just made up and sounds right but is entirely fiction. Confabulation is how many people explain their own behavior when questioned, but also how many people get through interviews, or make business deals, or mansplain …
-
@RnDanger @elfburgerman @gotofritz @AccordionBruce @grammargirl Exactly. Making machines seem like magic, seem like they have no internal mechanism, is a common tactic. It's why we refer to external hard drives that we don't own or control as "the cloud."
@orionkidder @RnDanger @elfburgerman @AccordionBruce @grammargirl
Sounds all a bit conspiracy theory to me.
There is nothing positive about "hallucinating", I wouldn't ride a bus if I knew the driver was prone to hallucinating
-
@orionkidder @RnDanger @elfburgerman @AccordionBruce @grammargirl
Sounds all a bit conspiracy theory to me.
There is nothing positive about "hallucinating", I wouldn't ride a bus if I knew the driver was prone to hallucinating
@gotofritz @RnDanger @elfburgerman @AccordionBruce @grammargirl It's a marketing tactic.
And the problem with the metaphor of hallucination was explained at the top of the thread.
I'll be blocking you if you keep playing ignorant.
-
The word "hallucination" isn't going away — it's a widely used industry term — but we need to explain it better for beginners:
"Hallucination" is just a fancy word for "confidently makes mistakes":
"Remember: AI hallucinates, and you need to confirm all facts" should be something like "Remember: AI confidently makes mistakes, and you need to confirm all facts" or "AI tells you things that are wrong in a way that sounds completely believable. Confirm all facts!"
none of the people using this word have ever experienced an hallucination
-
@grammargirl This is a good example of why that term is so dangerous. Thank you for posting it.
That said, while I have zero hope of making that term go away, we also have the word "slop" as a counter.
"Ugh. It had a hallucination..."
"Yup. And the results are now slop."
That said, I don't myself use "hallucination" in the "AI" context. I refer to the error rate, which last I checked, hovered around 40%.
@orionkidder @grammargirl I refuse to use it and anthropomorphize a computer failing.
It’s like “pro life.” No, it’s “give birth or die trying.” We need to call things what they are.
-
The word "hallucination" isn't going away — it's a widely used industry term — but we need to explain it better for beginners:
"Hallucination" is just a fancy word for "confidently makes mistakes":
"Remember: AI hallucinates, and you need to confirm all facts" should be something like "Remember: AI confidently makes mistakes, and you need to confirm all facts" or "AI tells you things that are wrong in a way that sounds completely believable. Confirm all facts!"
I have always liked Harry Frankfurt's definition of 'bullshit' as the boasting of a statement, one utterly without regard for its truth or falsity, but delivered with outright certainty.
So 'hallucination' might be the industry-standard lie, but I've simply come to call these machines 'bullshit engines'. Whether they are right doesn't matter, but they sure do sound like they know...
-
The word "hallucination" isn't going away — it's a widely used industry term — but we need to explain it better for beginners:
"Hallucination" is just a fancy word for "confidently makes mistakes":
"Remember: AI hallucinates, and you need to confirm all facts" should be something like "Remember: AI confidently makes mistakes, and you need to confirm all facts" or "AI tells you things that are wrong in a way that sounds completely believable. Confirm all facts!"
@grammargirl It’s a fancy-sounding excuse for having precision issues with floating-point math, which they cover by adding randomness and calling it a feature (“temperature”).
-
The word "hallucination" isn't going away — it's a widely used industry term — but we need to explain it better for beginners:
"Hallucination" is just a fancy word for "confidently makes mistakes":
"Remember: AI hallucinates, and you need to confirm all facts" should be something like "Remember: AI confidently makes mistakes, and you need to confirm all facts" or "AI tells you things that are wrong in a way that sounds completely believable. Confirm all facts!"
@grammargirl Totally agree, but I think it's more complicated than replacing "hallucinates" with another wording like "makes mistakes confidently". For some people answering confidently equals knowing. A genuine, friendly manner equals trustworthiness. Coherence equals truth. They're judging doesn't rely on fact-checking information but on communication behavior. And these machines are quite good in mimicking human communication.
