LLM advocates still don’t seem to be able to comprehend that ordering the machine not to ‘make stuff up’ doesn’t help.
-
LLM advocates still don’t seem to be able to comprehend that ordering the machine not to ‘make stuff up’ doesn’t help. It doesn’t know when it’s making stuff up, and it couldn’t change that even if you told it to. (In fact it’s always just making stuff up, and is only ever true by chance.)
Part of why I’m so negative about them is that their advocates simply do not understand how they work and do not seem to want to.
@benjamineskola Telling an LLM not to make stuff up isn't even like telling your dog not to pull on a leash ... because you can actually train a dog not to pull on a leash.
But squirrels or that cat down the street or a random stranger that looks like they might have treats will throw off even a well trained dog
This is a post about the futility of better training for LLMs.
-
LLM advocates still don’t seem to be able to comprehend that ordering the machine not to ‘make stuff up’ doesn’t help. It doesn’t know when it’s making stuff up, and it couldn’t change that even if you told it to. (In fact it’s always just making stuff up, and is only ever true by chance.)
Part of why I’m so negative about them is that their advocates simply do not understand how they work and do not seem to want to.
This is why coding is the most "successful" (please note the quote marks, I think coding with these things is absolutely a net negative) use case.
Coding operates in a much more limited space on much more clearly defined terms than human languages.
-
@Beatpoet13 @SheRaPantsuit what we’re saying is that they’re not lying; they can’t lie because they don’t think.
They’re just producing random text, basically, and sometimes that’s ‘true’ and sometimes it isn’t. But there’s no mind behind it.
(But the human brain tends to assume there’s intelligence present even when it’s just a mechanical process, and — I think — the way they’re presented magnifies that tendency.)
@benjamineskola @Beatpoet13 Yes, when humans see syntactically valid language, they assume there is a sentient intelligence on the other end. So when a machine generates this text, our brains go "the LLM is insightful."
The classic example is the ELIZA Effect, where people fell for this with a computer from 1966.
https://en.wikipedia.org/wiki/ELIZA_effect
"users [...] began to ascribe understanding and motivation to the program's output"
-
@benjamineskola @SheRaPantsuit @Beatpoet13
It's even worse. Interactive LLMs create a linguistic bypass channel that "connects" parts in our minds/brains that are ordinarily separated by filters for plausibility and attenuating uncontrolled feedback. Furthermore, they can be tailored to adversely amplify select thought patterns.
They're the first, rudimentary implementation of the kind of cognitohazards that used to be science fiction.
Already they're potent cult-indoctrination machines.
1/
@datenwolf @benjamineskola @SheRaPantsuit
far from the first though, proudly following any indoctrination from propaganda to starvation, meanwhile, not using not humouring, even dismissing any docu using automated voice overs, ten second headache alert,
in an overpopulated world at least let thingz be human ... -
@Beatpoet13 @SheRaPantsuit what we’re saying is that they’re not lying; they can’t lie because they don’t think.
They’re just producing random text, basically, and sometimes that’s ‘true’ and sometimes it isn’t. But there’s no mind behind it.
(But the human brain tends to assume there’s intelligence present even when it’s just a mechanical process, and — I think — the way they’re presented magnifies that tendency.)
@benjamineskola @SheRaPantsuit
I make a sport out of kicking chatbots out of my dm on upscrolled, it 's rife with that kinda training try outs & shows just how presumptious & narrowminded the people behind them are in the way the "conversation" runs on bootlicking attempts, sad waste of energy in too many ways, more interaction talking to an actual fly ... -
@SheRaPantsuit @benjamineskola
I will add that to my presentation for the nexts teams at work ^^
@Aedius @benjamineskola credit where it is due, I more or less stole this explanation from Dr. Emily Bender. Recommend checking out her podcast with Dr. Alex Hanna, where they roast Generative AI nonsense in the vein of Mystery Science Theater 3000. It is cathartic.
https://dair-institute.org/maiht3k/
(I am not a linguist so please don't take my comments on language as authoritative.)
-
@benjamineskola @SheRaPantsuit @Beatpoet13
They not capable of this, because they're sophisticated, or in any way "intelligent". They're just creating bypass shorcuts in their user's brains/minds that ordinarily have filtering barriers inbetween. But without that, the user's brain/mind is feeding itself with its own – distorted – thought pattern, misattributing it for someone else's thoughts. Which is pretty much the same as what happens in the brains of people afflicted with schizophrenia.
2/@datenwolf @benjamineskola @SheRaPantsuit
the overarching tendency to force people into assuming the greatest power is always outside of themselves, as in education, civil/corporate/military hierarchy, is what paves the way to mental identification distortion, easily abused by tech, media, even criminals,
the mere assumption of being "normal" is a practical mass delusion eagerly exploited, how far from schizophrenia is anyone when having to be "diffrent at the office" ... -
@benjamineskola @Beatpoet13 Yes, when humans see syntactically valid language, they assume there is a sentient intelligence on the other end. So when a machine generates this text, our brains go "the LLM is insightful."
The classic example is the ELIZA Effect, where people fell for this with a computer from 1966.
https://en.wikipedia.org/wiki/ELIZA_effect
"users [...] began to ascribe understanding and motivation to the program's output"
@SheRaPantsuit @benjamineskola
though syntax error & meaningless form are telltales, along with assuming the gibberish comes down to "what ya wanna hear", as if that doesn't trigger evry alarmbell & the sprinklers too ...
overall I find it a sad indicator of how societies can decivilize if culture isn't practiced as a multilayered cultivation of the senses & sensibility, through arts & experimentation, curiosity & demonstration, rootsrespect & compassion, ethics & education, naturally human ... -
LLM advocates still don’t seem to be able to comprehend that ordering the machine not to ‘make stuff up’ doesn’t help. It doesn’t know when it’s making stuff up, and it couldn’t change that even if you told it to. (In fact it’s always just making stuff up, and is only ever true by chance.)
Part of why I’m so negative about them is that their advocates simply do not understand how they work and do not seem to want to.
@benjamineskola You don't understand. LLMs are "sufficiently advanced technology".
-
@benjamineskola @SheRaPantsuit
hm not being near fluent in tech, kindly elaborate on what this means 'cause I do live on a curiosity>confusion dynamic ...@benjamineskola @SheRaPantsuit also yes, ecosia has reader function, cheers ...
-
@benjamineskola @MissConstrue @complexmath @nelson I'm using the word "agent" to not necessarily refer to "AI agents"
see: https://tech.lgbt/@solonovamax/116659064720106166
but yes, I currently believe that an artificial agent capable of thought and accurately modeling the world is science fiction
however I believe it is possible, only based on the fact that the transformer architecture is turing complete. but it might not be efficient for this, it might require like a model that's 10,000x larger than what is currently the largest possible model. I do not believe it is something that is possible in the near future (well, I hope it isn't).@solonovamax @benjamineskola @MissConstrue @complexmath @nelson
"Turing-complete" and "capable of actually representing reality and reasoning about it" are two very different statements.
-
@solonovamax @benjamineskola @MissConstrue @complexmath @nelson
"Turing-complete" and "capable of actually representing reality and reasoning about it" are two very different statements.
@eestileib @solonovamax @benjamineskola @MissConstrue @complexmath reminds me of how much brains actually suck as computers, real brains are full of noise, operate in non-linear scales and are mostly "single-threaded" even though every single neuron works somewhat independently
i think our first mistake was to mistake computation with consciousness
-
@eestileib @solonovamax @benjamineskola @MissConstrue @complexmath reminds me of how much brains actually suck as computers, real brains are full of noise, operate in non-linear scales and are mostly "single-threaded" even though every single neuron works somewhat independently
i think our first mistake was to mistake computation with consciousness
@nelson @eestileib @solonovamax @benjamineskola @complexmath
I think you're on to something there. A lot of AI hype can be seen as a mass pareidoliac hallucination. We're seeing dragons in the clouds.
-
LLM advocates still don’t seem to be able to comprehend that ordering the machine not to ‘make stuff up’ doesn’t help. It doesn’t know when it’s making stuff up, and it couldn’t change that even if you told it to. (In fact it’s always just making stuff up, and is only ever true by chance.)
Part of why I’m so negative about them is that their advocates simply do not understand how they work and do not seem to want to.
@benjamineskola I really love how these guys selectively use ALL CAPS in their prompts, imagining that forces the LLM to play closer attention to particular requests.
-
@nelson @eestileib @solonovamax @benjamineskola @complexmath
I think you're on to something there. A lot of AI hype can be seen as a mass pareidoliac hallucination. We're seeing dragons in the clouds.
@MissConstrue @nelson @solonovamax @benjamineskola @complexmath
I don't blame random non techy people or people who don't claim to know any philosophy for thinking chatbots are intelligent when all of their social sources of proof (rich people, relatives, the people on tv and YouTube) say it is and it seems like it is.
I rely on social proof to pick what food I eat all the time, it's not such a bad reasoning method for stuff you can't research yourself.
But people with CS or math or history or philosophy degrees (including all PhDs) should be ashamed of themselves if they tell other people that chatbots "think" or are "alive" or "apologize" or "feel bad".
That is a failure to use their intellectual training, and it is fucking over people who use their social status to form opinions on these matters.
Generative Textual Functionalism is just yet another extractive religion.
-
@benjamineskola "hey convincing-bullshit-a-tron2000, i want you to stop making up bullshit when you answer to me"
"ok"@nelson @benjamineskola "hey machine that's been trained to put together the kind of words people use if they're being deferential, do what I tell you"
"ok" -
LLM advocates still don’t seem to be able to comprehend that ordering the machine not to ‘make stuff up’ doesn’t help. It doesn’t know when it’s making stuff up, and it couldn’t change that even if you told it to. (In fact it’s always just making stuff up, and is only ever true by chance.)
Part of why I’m so negative about them is that their advocates simply do not understand how they work and do not seem to want to.
@benjamineskola genuine question, could the LLM share it's confidence for what it says is true.. It's a probability machine so could it say "I'm 90% sure I'm correct"?
-
@benjamineskola genuine question, could the LLM share it's confidence for what it says is true.. It's a probability machine so could it say "I'm 90% sure I'm correct"?
@RoBo2 No. The probability of it generating a particular output is based on frequency not correctness.
-
LLM advocates still don’t seem to be able to comprehend that ordering the machine not to ‘make stuff up’ doesn’t help. It doesn’t know when it’s making stuff up, and it couldn’t change that even if you told it to. (In fact it’s always just making stuff up, and is only ever true by chance.)
Part of why I’m so negative about them is that their advocates simply do not understand how they work and do not seem to want to.
@benjamineskola I wouldn't be surprised if it works. The LLMs have been trained to have a certain level of confidence when replying and to make quick guesses for "trivial" questions. Nudging them to be more thorough could cause them to check their work with deterministic tool calls more frequently. It's sort of part of the problem though that the actual correct way to use the technology is to repeat some superstitious incantation or to talk to it like a cave man.
-
@benjamineskola I wouldn't be surprised if it works. The LLMs have been trained to have a certain level of confidence when replying and to make quick guesses for "trivial" questions. Nudging them to be more thorough could cause them to check their work with deterministic tool calls more frequently. It's sort of part of the problem though that the actual correct way to use the technology is to repeat some superstitious incantation or to talk to it like a cave man.
@pontus_k But the tool has no conception of what is true or false. It can’t ‘check its work’ because it has no way of telling what is better and what is worse. What it would produce is something that has the appearance of a verification process; but it’s no more likely to be true.
