LLM advocates still don’t seem to be able to comprehend that ordering the machine not to ‘make stuff up’ doesn’t help.
-
@benjamineskola
It’s not “hallucinations”.
It’s just putting a stream of words together on the specified subject in a SYNTACTICALLY correct order.
Nothing more, nothing less.
Semantics, accuracy, TRUTH don’t even enter into it.
FFS, it’s not “intelligent”. It’s code.
“Computers don’t make mistakes.” (remember that one?)
People make mistakes. People program computers. (At least, they used to.)
Cthulhu save us all.@mysturji Yes, that is my point.
-
@benjamineskola for example if I typed "the cat sat on the " would it work out the probably the next word is "mat" with a score of 87%
@RoBo2 Yes: probability. The sentence is a common one, so it’s likely to be reproduced in the output. But the LLM has no conception of whether a cat really did sit on the mat.
You probably could build an LLM so that it showed the probabilities of each token; but it wouldn’t solve the problem being discussed here at all.
-
@pontus_k But the tool has no conception of what is true or false. It can’t ‘check its work’ because it has no way of telling what is better and what is worse. What it would produce is something that has the appearance of a verification process; but it’s no more likely to be true.
@benjamineskola Some of these systems have access to deterministic tools that could give them a better output. For example, all LLMs struggle with counting letters, but in a lot of cases they have the capability to call the unix utility 'wc' to count letters. Putting 'MAKE NO MISTAKES' in the prompt could possibly make it a bit more likely that it does so and gets it right. Don't get me wrong, I think it's absolutely stupid that this is where we are.
-
LLM advocates still don’t seem to be able to comprehend that ordering the machine not to ‘make stuff up’ doesn’t help. It doesn’t know when it’s making stuff up, and it couldn’t change that even if you told it to. (In fact it’s always just making stuff up, and is only ever true by chance.)
Part of why I’m so negative about them is that their advocates simply do not understand how they work and do not seem to want to.
@benjamineskola Colleague who is adding "AGENTS.md" files to our repositories is adding very similar paragraphs to those files.
Ugh.
-
@benjamineskola Some of these systems have access to deterministic tools that could give them a better output. For example, all LLMs struggle with counting letters, but in a lot of cases they have the capability to call the unix utility 'wc' to count letters. Putting 'MAKE NO MISTAKES' in the prompt could possibly make it a bit more likely that it does so and gets it right. Don't get me wrong, I think it's absolutely stupid that this is where we are.
@pontus_k you don’t need to hunt for ways to make this make sense.
-
@benjamineskola Colleague who is adding "AGENTS.md" files to our repositories is adding very similar paragraphs to those files.
Ugh.
@juliancalaby That sort of thing bugs me so much. Like, if you insist on using these tools (and I know I'm not going to win the fight against them more generally), then at least use them properly.
I've tried to have conversations about 'how do we know whether this actually makes a difference' and so on, and I think it's probably better than it could be, but it's still very silly.
-
@juliancalaby That sort of thing bugs me so much. Like, if you insist on using these tools (and I know I'm not going to win the fight against them more generally), then at least use them properly.
I've tried to have conversations about 'how do we know whether this actually makes a difference' and so on, and I think it's probably better than it could be, but it's still very silly.
@benjamineskola I wrote our company's AI policy, added terms to require short- and long-term evaluation of whether this is actually working for us, and management as a whole agreed so it's company policy. Which is a nice. However the head of the company has gone very AI and is pulling the company in that direction despite ... well ... all the clear points against it and the person who is functionally our sysadmin is heading up a project to add it into our workflows and is using it to do stuff with our infrastructure.
I'm now trying to keep them accountable and biding my time before this blows up in their faces.
Thankfully the "accountability" story is working out fairly well so far, but it's fucking exhausting dealing with this bullshit.
-
@benjamineskola The problem with Obasanjo is he's utterly unprincipled and just chasing engagement/self-aggrandizement. His purpose for being in social spaces like Masto/Bluesky/X is to stroke his ego, so everything he does is just an act of public masturbation.
He's interested in self-aggrandizement and self-promotion, nothing more.
Which is why I use the descriptor "piece of shit" with regard to him.
@prietschka @benjamineskola It’s refreshing to see people stating this plainly. These people are dumb, they make bad choices, and making that observation is not mean.
-
LLM are a Skinner Box.
@Aedius @skotchygut @benjamineskola
Designed by wannabe supervillains, built with wartime-scale resources.
The hell do people expect??
-
@nelson @benjamineskola @solonovamax
yeah, i think my take from about a year ago still mostly holds up https://biplus.social/@linkplay/114828181247605258@linkplay @nelson @benjamineskola @solonovamax
It's just rolling linguistic dice, words bouncing around between probablistic paddles in a bigass pachinko matrix. It's not designed to vet facts. It's designed to regurgitate plausible spitwads that RESEMBLE facts. And the weights behind all those paddles and slots are tuned to whatever agenda the designers wish.
...And the designers serve planetwrecking technofascist war profiteers who party with people like Epstein.
Why would anyone ever trust it with so much as a goddamn casserole recipe??
-
@Su_G @Aedius @benjamineskola I shamelessly stole the term from @davidgerard
-
LLM advocates still don’t seem to be able to comprehend that ordering the machine not to ‘make stuff up’ doesn’t help. It doesn’t know when it’s making stuff up, and it couldn’t change that even if you told it to. (In fact it’s always just making stuff up, and is only ever true by chance.)
Part of why I’m so negative about them is that their advocates simply do not understand how they work and do not seem to want to.
@benjamineskola
Even if you curate the source text in detail, there is no guarantee that you avoid falsehoods and imaginings. When you just feed the entire WWW blindly into your LLM, of course all bets are off. -
@benjamineskola
Even if you curate the source text in detail, there is no guarantee that you avoid falsehoods and imaginings. When you just feed the entire WWW blindly into your LLM, of course all bets are off.@mrundkvist yes exactly. It’s a complete misunderstanding of how these tools work.
-
@mrundkvist yes exactly. It’s a complete misunderstanding of how these tools work.
@benjamineskola
Also a breathtakingly naïve idea about truth! -
LLM advocates still don’t seem to be able to comprehend that ordering the machine not to ‘make stuff up’ doesn’t help. It doesn’t know when it’s making stuff up, and it couldn’t change that even if you told it to. (In fact it’s always just making stuff up, and is only ever true by chance.)
Part of why I’m so negative about them is that their advocates simply do not understand how they work and do not seem to want to.
@benjamineskola Part of why I'm so frustrated is that their detractors simply do not understand how they work and do not seem to want to.
(No, they're not simply correct "by chance")
/actual very very senior greybeard dev who challenged his own erroneous convictions regarding LLMs and now find them useful
-
@benjamineskola Part of why I'm so frustrated is that their detractors simply do not understand how they work and do not seem to want to.
(No, they're not simply correct "by chance")
/actual very very senior greybeard dev who challenged his own erroneous convictions regarding LLMs and now find them useful
@troed whatever you feel the need to tell yourself to justify it.
-
@troed whatever you feel the need to tell yourself to justify it.
@benjamineskola Do you ever ponder that you might be wrong about something?
-
@benjamineskola Do you ever ponder that you might be wrong about something?
@troed all the time! But that doesn’t mean I’m wrong here.
-
@troed all the time! But that doesn’t mean I’m wrong here.
@benjamineskola You are. Interested in learning?
(I was wrong about LLMs for development, took the time to learn, and changed my mind)
-
@benjamineskola You are. Interested in learning?
(I was wrong about LLMs for development, took the time to learn, and changed my mind)
@troed I’m not really interested in hearing your justifications for why actually it makes total sense to just tell the text generator ‘don’t make stuff up!’ as if it’s doing so by choice.
