I've never been opposed to the word "hallucinating" for describing how AI makes mistakes ... until now.
-
@grammargirl I don’t think we need to accept it just yet. The word is deceptive—intentionally so. What needs to be explained is this: chatbots and LLMs can't "hallucinate” because they have no minds or senses. They routinely depart from factuality because that's how they’re programmed: to generate plausible streams of text without regard to reality. (https://around.com/dont-trust-them/)
the consistent trend of anthropomorphizing badly written programs, and the machines the programs run on, is used to make tech CEO's as a religious ruling class.
they create these facsimiles of truth and reality then prop themselves up as the sole interpreters and arbiters. like any religious hierarchy.
they're relying on humans ingrained need to assign importance to random objects and events and an interpreter to hand out judgement in return for taking all their money.
-
The word "hallucination" isn't going away — it's a widely used industry term — but we need to explain it better for beginners:
"Hallucination" is just a fancy word for "confidently makes mistakes":
"Remember: AI hallucinates, and you need to confirm all facts" should be something like "Remember: AI confidently makes mistakes, and you need to confirm all facts" or "AI tells you things that are wrong in a way that sounds completely believable. Confirm all facts!"
@grammargirl these folks are stealing language to whitewash a con. In my opinion.
Hallucination is a deviation from the normal way healthy human minds work. The confident incorrectness presented by the companies shilling AI is working as designed.
-
The word "hallucination" isn't going away — it's a widely used industry term — but we need to explain it better for beginners:
"Hallucination" is just a fancy word for "confidently makes mistakes":
"Remember: AI hallucinates, and you need to confirm all facts" should be something like "Remember: AI confidently makes mistakes, and you need to confirm all facts" or "AI tells you things that are wrong in a way that sounds completely believable. Confirm all facts!"
@grammargirl But what actually is the point of using it if I have to confirm all facts? Can’t I just skip the middleman?
-
The word "hallucination" isn't going away — it's a widely used industry term — but we need to explain it better for beginners:
"Hallucination" is just a fancy word for "confidently makes mistakes":
"Remember: AI hallucinates, and you need to confirm all facts" should be something like "Remember: AI confidently makes mistakes, and you need to confirm all facts" or "AI tells you things that are wrong in a way that sounds completely believable. Confirm all facts!"
@grammargirl like when medical people call someone "confused", AI "hallucination" is a more precise term than common parlance. it basically means the bot couldn't find a plausible answer and is for some reason blocked from saying "I don't know", so it makes stuff up.
that's a bit different from "confidently makes mistakes" becuase it's "confidently making stuff up entirely".
I have no idea what would be a good replacement for "hallucinate" in this context, I agree that it feels deceptive as is though.
-
@grammargirl I don’t think we need to accept it just yet. The word is deceptive—intentionally so. What needs to be explained is this: chatbots and LLMs can't "hallucinate” because they have no minds or senses. They routinely depart from factuality because that's how they’re programmed: to generate plausible streams of text without regard to reality. (https://around.com/dont-trust-them/)
IMO "confabulation" is more accurate than "hallucination" because the former indicates a lack of intent. Given that LLMs are not sentient, they lack intention. At most, they are reflexively responding to a reward function that optimizes towards producing text roughly resembling the pattern of their training data, but that's different from intent.
-
I've never been opposed to the word "hallucinating" for describing how AI makes mistakes ... until now.
I just talked to someone who thought AI hallucinations would be obvious because it would be obvious if you talked to a *person* who was hallucinating.
In other words, they equated "hallucination" with "sounds wacko" and accepted AI output as true because it sounded level headed.
1/2
@grammargirl I have a general dislike to use terms related to human cognition for anything AI does. There are already enough loonies out there who think a system throwing way too many dice could be their friend or is anything more than the dice. We need other/better terms for that.
-
I've never been opposed to the word "hallucinating" for describing how AI makes mistakes ... until now.
I just talked to someone who thought AI hallucinations would be obvious because it would be obvious if you talked to a *person* who was hallucinating.
In other words, they equated "hallucination" with "sounds wacko" and accepted AI output as true because it sounded level headed.
1/2
@grammargirl I oppose the term because it implies that LLMs are conscious, and occasionally have “experiences” that aren’t true, instead of being just text-generating software that outputs incorrect information.
-
The word "hallucination" isn't going away — it's a widely used industry term — but we need to explain it better for beginners:
"Hallucination" is just a fancy word for "confidently makes mistakes":
"Remember: AI hallucinates, and you need to confirm all facts" should be something like "Remember: AI confidently makes mistakes, and you need to confirm all facts" or "AI tells you things that are wrong in a way that sounds completely believable. Confirm all facts!"
@grammargirl I agree it’s not going away. I still find it constructive to point out it’s misleading, though, because it’s a good framing device for talking about what these technologies are and are not actually doing.
-
@grammargirl But what actually is the point of using it if I have to confirm all facts? Can’t I just skip the middleman?
@feisty_lemming It depends on what you're using it for. If you're fact checking, it can be faster to put in a document and say something like "Fact check this piece. Show your sources," which gives you a list of links to click and check. It's faster than putting each thing you want to check into Google and then sorting through the links (and now the AI slop too). It will also surface relevant links you may have missed that don't show up in the first 10 or 20 on Google.
-
@feisty_lemming It depends on what you're using it for. If you're fact checking, it can be faster to put in a document and say something like "Fact check this piece. Show your sources," which gives you a list of links to click and check. It's faster than putting each thing you want to check into Google and then sorting through the links (and now the AI slop too). It will also surface relevant links you may have missed that don't show up in the first 10 or 20 on Google.
@feisty_lemming You can also specify the sources you want it to use with something like "These are the 20 sites I usually use. Check there first and add anything else that seems relevant."
But I'm sure there are lots of other use cases where it's more in the way than helpful.
-
@grammargirl As a mental health nurse, I would say delusions would be a more accurate term than hallucinations. I would also point out as someone else said that unless you are trained to do so, you may not know someone is experiencing hallucinations. Many who experience them chronically are quite good at presenting as if they aren't.
@Burnt_Veggies Thanks!
-
@feisty_lemming It depends on what you're using it for. If you're fact checking, it can be faster to put in a document and say something like "Fact check this piece. Show your sources," which gives you a list of links to click and check. It's faster than putting each thing you want to check into Google and then sorting through the links (and now the AI slop too). It will also surface relevant links you may have missed that don't show up in the first 10 or 20 on Google.
@grammargirl Maybe it would be faster. I object to the mass illegality of the content theft, the environmental destruction, and all the other terrible things that come with it. So I can’t bring myself to use it in order to possibly do stuff faster. And I’m fortunate that for work at least, so far I’m not being forced to. Many who object are not that lucky.
-
@orionkidder @grammargirl
The explanation has to include that if you believe what the AI tells you then you are hallucinating@AccordionBruce @orionkidder @grammargirl
Exactly this.
Hallucination is an act of cognition. The machine doesn't -
@feisty_lemming You can also specify the sources you want it to use with something like "These are the 20 sites I usually use. Check there first and add anything else that seems relevant."
But I'm sure there are lots of other use cases where it's more in the way than helpful.
I've done that and it generates ballpark-but-not-accurate information with fake citations.
-
I've done that and it generates ballpark-but-not-accurate information with fake citations.
@eestileib Fake citations (and fake quotations) are a huge problem. And sometimes it’s not even that the citation is fully fake, but a real source has been transmogrified so the details are wrong—authors are in the wrong order, title is modified, etc. @grammargirl
-
@orionkidder Good point.
Also, the error rate now highly depends on which model you're talking about, but I think that's the rate for those that are most widely used -- e.g., the free models.
@grammargirl I'm seeing people claim the error rate is lower with other models, and I'm not sure I believe that since this industries just piles lies on top of lies, but the only plausible explanation of the lowered error rate I've seen is for Claude code.
-
@grammargirl I'm seeing people claim the error rate is lower with other models, and I'm not sure I believe that since this industries just piles lies on top of lies, but the only plausible explanation of the lowered error rate I've seen is for Claude code.
@grammargirl If I understand correctly, it shoves every query through the "AI" multiple times and tests whether it does the thing it's asked to do, but of course, it hides all of that from the user.
-
@grammargirl If I understand correctly, it shoves every query through the "AI" multiple times and tests whether it does the thing it's asked to do, but of course, it hides all of that from the user.
@grammargirl To me, that feels like a brute-force workaround, a kludge, not an improvement in the tech itself. It's like saying, my car is too slow, so I'll attach a second engine to the hood.
-
I've done that and it generates ballpark-but-not-accurate information with fake citations.
@eestileib @feisty_lemming I check everything and haven't had that problem. I find errors in maybe 1 in 50 links--like the page doesn't say what the model says it does--it's so rare that's just a total guess at the rate.
I'm not asking it to find new information -- just to check existing info. Not sure if that would be the difference. I also don't use the free models. They are dramatically worse.
-
The word "hallucination" isn't going away — it's a widely used industry term — but we need to explain it better for beginners:
"Hallucination" is just a fancy word for "confidently makes mistakes":
"Remember: AI hallucinates, and you need to confirm all facts" should be something like "Remember: AI confidently makes mistakes, and you need to confirm all facts" or "AI tells you things that are wrong in a way that sounds completely believable. Confirm all facts!"
@grammargirl Would "delusional" be more apt?
