I've never been opposed to the word "hallucinating" for describing how AI makes mistakes ... until now.
-
I've never been opposed to the word "hallucinating" for describing how AI makes mistakes ... until now.
I just talked to someone who thought AI hallucinations would be obvious because it would be obvious if you talked to a *person* who was hallucinating.
In other words, they equated "hallucination" with "sounds wacko" and accepted AI output as true because it sounded level headed.
1/2
The word "hallucination" isn't going away — it's a widely used industry term — but we need to explain it better for beginners:
"Hallucination" is just a fancy word for "confidently makes mistakes":
"Remember: AI hallucinates, and you need to confirm all facts" should be something like "Remember: AI confidently makes mistakes, and you need to confirm all facts" or "AI tells you things that are wrong in a way that sounds completely believable. Confirm all facts!"
-
The word "hallucination" isn't going away — it's a widely used industry term — but we need to explain it better for beginners:
"Hallucination" is just a fancy word for "confidently makes mistakes":
"Remember: AI hallucinates, and you need to confirm all facts" should be something like "Remember: AI confidently makes mistakes, and you need to confirm all facts" or "AI tells you things that are wrong in a way that sounds completely believable. Confirm all facts!"
@grammargirl This is a good example of why that term is so dangerous. Thank you for posting it.
That said, while I have zero hope of making that term go away, we also have the word "slop" as a counter.
"Ugh. It had a hallucination..."
"Yup. And the results are now slop."
That said, I don't myself use "hallucination" in the "AI" context. I refer to the error rate, which last I checked, hovered around 40%.
-
@grammargirl This is a good example of why that term is so dangerous. Thank you for posting it.
That said, while I have zero hope of making that term go away, we also have the word "slop" as a counter.
"Ugh. It had a hallucination..."
"Yup. And the results are now slop."
That said, I don't myself use "hallucination" in the "AI" context. I refer to the error rate, which last I checked, hovered around 40%.
@orionkidder @grammargirl
The explanation has to include that if you believe what the AI tells you then you are hallucinating -
@grammargirl This is a good example of why that term is so dangerous. Thank you for posting it.
That said, while I have zero hope of making that term go away, we also have the word "slop" as a counter.
"Ugh. It had a hallucination..."
"Yup. And the results are now slop."
That said, I don't myself use "hallucination" in the "AI" context. I refer to the error rate, which last I checked, hovered around 40%.
@orionkidder Good point.
Also, the error rate now highly depends on which model you're talking about, but I think that's the rate for those that are most widely used -- e.g., the free models.
-
I've never been opposed to the word "hallucinating" for describing how AI makes mistakes ... until now.
I just talked to someone who thought AI hallucinations would be obvious because it would be obvious if you talked to a *person* who was hallucinating.
In other words, they equated "hallucination" with "sounds wacko" and accepted AI output as true because it sounded level headed.
1/2
@grammargirl Hm. It’s not always obvious if a person you are talking to is hallucinating, depending on what their hallucinations are and what they say.
I get their point, just am sad on the mental illness rep side.
-
@grammargirl Hm. It’s not always obvious if a person you are talking to is hallucinating, depending on what their hallucinations are and what they say.
I get their point, just am sad on the mental illness rep side.
@queenofnewyork Yes, I was thinking about that/wondering about that as I was posting, but that is the common perception.
-
The word "hallucination" isn't going away — it's a widely used industry term — but we need to explain it better for beginners:
"Hallucination" is just a fancy word for "confidently makes mistakes":
"Remember: AI hallucinates, and you need to confirm all facts" should be something like "Remember: AI confidently makes mistakes, and you need to confirm all facts" or "AI tells you things that are wrong in a way that sounds completely believable. Confirm all facts!"
@grammargirl I don’t think we need to accept it just yet. The word is deceptive—intentionally so. What needs to be explained is this: chatbots and LLMs can't "hallucinate” because they have no minds or senses. They routinely depart from factuality because that's how they’re programmed: to generate plausible streams of text without regard to reality. (https://around.com/dont-trust-them/)
-
I've never been opposed to the word "hallucinating" for describing how AI makes mistakes ... until now.
I just talked to someone who thought AI hallucinations would be obvious because it would be obvious if you talked to a *person* who was hallucinating.
In other words, they equated "hallucination" with "sounds wacko" and accepted AI output as true because it sounded level headed.
1/2
@grammargirl
I prefer "bullshit" to "hallucination". It gets much closer to what's really happening, and it gives users a much better idea of how to deal with it. -
I've never been opposed to the word "hallucinating" for describing how AI makes mistakes ... until now.
I just talked to someone who thought AI hallucinations would be obvious because it would be obvious if you talked to a *person* who was hallucinating.
In other words, they equated "hallucination" with "sounds wacko" and accepted AI output as true because it sounded level headed.
1/2
@grammargirl Tools are useful, but they are never a substitute for human judgement. It's like people who write something and accept corrections from spell check without thinking about what the words mean.
The most powerful tools are most powerful in the hands of those who can validate correctness quickly. We're still responsible for validating all facts, regardless of source. We can't outsource critical thinking to machines, which do not think at all.
-
The word "hallucination" isn't going away — it's a widely used industry term — but we need to explain it better for beginners:
"Hallucination" is just a fancy word for "confidently makes mistakes":
"Remember: AI hallucinates, and you need to confirm all facts" should be something like "Remember: AI confidently makes mistakes, and you need to confirm all facts" or "AI tells you things that are wrong in a way that sounds completely believable. Confirm all facts!"
@grammargirl
Hallucinations can only happen to a mind. An LLM has no more mind than a slot machine.The people making this stuff fell in love with their own convincing automatons, so attributed ‘hallucination’ as happening to their little babies.
It’s a much different thing if you say ‘this brainless machine is constantly making errors and spitting incorrect data’.
If you say that, it means back to the drawing board: this demo tech has failed.
-
I've never been opposed to the word "hallucinating" for describing how AI makes mistakes ... until now.
I just talked to someone who thought AI hallucinations would be obvious because it would be obvious if you talked to a *person* who was hallucinating.
In other words, they equated "hallucination" with "sounds wacko" and accepted AI output as true because it sounded level headed.
1/2
@grammargirl "Delusion" would have been better, but the ship has sailed.
-
@grammargirl Hm. It’s not always obvious if a person you are talking to is hallucinating, depending on what their hallucinations are and what they say.
I get their point, just am sad on the mental illness rep side.
@queenofnewyork @grammargirl
I think the film "A Beautiful Mind" brought home to me just how real hallucinations are to those who are experience them.
Which I hadn't previously appreciated. -
The word "hallucination" isn't going away — it's a widely used industry term — but we need to explain it better for beginners:
"Hallucination" is just a fancy word for "confidently makes mistakes":
"Remember: AI hallucinates, and you need to confirm all facts" should be something like "Remember: AI confidently makes mistakes, and you need to confirm all facts" or "AI tells you things that are wrong in a way that sounds completely believable. Confirm all facts!"
@grammargirl
The industry named its own mistakes ‘hallucinations.’
Hallucinations is a forgiving term.
‘Delusions’ would be more accurate. -
@grammargirl
I prefer "bullshit" to "hallucination". It gets much closer to what's really happening, and it gives users a much better idea of how to deal with it.@VATVSLPR @grammargirl you’re backed up by academia https://link.springer.com/article/10.1007/s10676-024-09775-5
-
I've never been opposed to the word "hallucinating" for describing how AI makes mistakes ... until now.
I just talked to someone who thought AI hallucinations would be obvious because it would be obvious if you talked to a *person* who was hallucinating.
In other words, they equated "hallucination" with "sounds wacko" and accepted AI output as true because it sounded level headed.
1/2
@grammargirl As a mental health nurse, I would say delusions would be a more accurate term than hallucinations. I would also point out as someone else said that unless you are trained to do so, you may not know someone is experiencing hallucinations. Many who experience them chronically are quite good at presenting as if they aren't.
-
I've never been opposed to the word "hallucinating" for describing how AI makes mistakes ... until now.
I just talked to someone who thought AI hallucinations would be obvious because it would be obvious if you talked to a *person* who was hallucinating.
In other words, they equated "hallucination" with "sounds wacko" and accepted AI output as true because it sounded level headed.
1/2
"Confabulation", although that is still anthropomorphism and implies agency which doesn't exist.
-
I've never been opposed to the word "hallucinating" for describing how AI makes mistakes ... until now.
I just talked to someone who thought AI hallucinations would be obvious because it would be obvious if you talked to a *person* who was hallucinating.
In other words, they equated "hallucination" with "sounds wacko" and accepted AI output as true because it sounded level headed.
1/2
I've been "opposed" to that (to the extent that it ever makes sense to oppose established language usage) from the start, favoring "fabrication".
All LLMs fabricate sentence-like objects which invite the user to hallucinate some meaning into them. Users then react to how they feel about the meaning that they themselves have imposed.
If one recognizes that all of the outputs are fabrications and that some fraction of them may be useful, one can go sanely about one's business.
-
I've never been opposed to the word "hallucinating" for describing how AI makes mistakes ... until now.
I just talked to someone who thought AI hallucinations would be obvious because it would be obvious if you talked to a *person* who was hallucinating.
In other words, they equated "hallucination" with "sounds wacko" and accepted AI output as true because it sounded level headed.
1/2
@grammargirl @Cdespinosa I propose the term “botsplaining”
-
@grammargirl I don’t think we need to accept it just yet. The word is deceptive—intentionally so. What needs to be explained is this: chatbots and LLMs can't "hallucinate” because they have no minds or senses. They routinely depart from factuality because that's how they’re programmed: to generate plausible streams of text without regard to reality. (https://around.com/dont-trust-them/)
the consistent trend of anthropomorphizing badly written programs, and the machines the programs run on, is used to make tech CEO's as a religious ruling class.
they create these facsimiles of truth and reality then prop themselves up as the sole interpreters and arbiters. like any religious hierarchy.
they're relying on humans ingrained need to assign importance to random objects and events and an interpreter to hand out judgement in return for taking all their money.
-
The word "hallucination" isn't going away — it's a widely used industry term — but we need to explain it better for beginners:
"Hallucination" is just a fancy word for "confidently makes mistakes":
"Remember: AI hallucinates, and you need to confirm all facts" should be something like "Remember: AI confidently makes mistakes, and you need to confirm all facts" or "AI tells you things that are wrong in a way that sounds completely believable. Confirm all facts!"
@grammargirl these folks are stealing language to whitewash a con. In my opinion.
Hallucination is a deviation from the normal way healthy human minds work. The confident incorrectness presented by the companies shilling AI is working as designed.
