Worth looking at both the quoted text here and •especially• the linked page, which is quite good.
-
It’s just •astonishing• how many eye-popping stories of LLMs doing amazing omg-verge-of-magical-superintelligence things turn out to be just unvarnished plagiarism.
In the first months after ChatGPT’s release, I remember a French dept colleague being amazed that GPT could translate and summarize a passage of Le Petit Prince.
It was a lot less impressive when I dug up the 2 or 3 online passages which it had copied almost verbatim and stitched together (sprinkling in a couple of extra words that made it less accurate).
3/
@inthehands (Completely aside: I sometimes feel like I ought to learn French *just* to be able to read that one in its original language. It is very special to me; it was the first book I ever read myself.
Even more completely aside: Someone ought to give Hayao Miyazaki the rights to create an animated interpretation of it, and supply him with however much coffee and pure adrenochrome needed to keep him alive, active and in good spirits for however long that takes. And reward him with whatever he wanted - for example, the exclusive right to hunt Sam Altman for sport.)
-
To be fair, competent plagiarism is a nontrivial task. It takes skill for a human to do: finding relevant fragments from an input dataset and stitching those fragments together in a well-formed way is not nothing! It’s impressive and interesting that people have figured out how to make machines do it.
But •that• is not the promise on which investors are valuing the AI megajuggernaut at trillions of dollars.
4/
The reasons we consider plagiarism to be cheating in school and malpractice in professional contexts are many of the same reasons that LLMs are not going to replace all knowledge-based human labor. (Details left as an exercise for the reader.)
And yes, a whole lot of what LLMs do •would• count as plagiarism if one of my students did it manually, and •should• count as plagiarism just the same if they use a machine to do it — not just in a “that’s cheating!!” sense, but more importantly in a “that’s not really doing the work” sense.
5/
-
Many years ago, the following idea spread in the art community: a 15 minutes doodle is the result of years of practice and study. In other words, the 15 minutes doodle took your entire life to make.
The same goes to LLMs: it did not take 90 minutes. It took the thousands of hours all the other softwares took to make, plus the years of study of all the programmers making them. And the thousands of hours manually labeling source codes...
@Lily_and_frog @inthehands I think that has been said for plumbing—$1 for the part, $45 for the five minutes it took to install the part, $400 for the years of experience it takes to know which part and how to install it.
-
RE: https://unstable.systems/@jneen/116618931097778342
Worth looking at both the quoted text here and •especially• the linked page, which is quite good.
I’ll add another item of my own. The first screenshot mentions giving an LLM the task of “implementing an HTTP server in JavaScript from scratch” in 90 minutes. Sounds impressive, right? Until you remember that every open-source Javascript HTTP server in existence ••was in the training data••.
1/
@inthehands it annoys me so much when people cite AI benchmarks for exactly this reason.
Either the logic of the benchmarks are private (and thus not really fit for judgement) or they're public in a way that effectively makes the supposed challenge of that benchmark into an open-book test. -
RE: https://unstable.systems/@jneen/116618931097778342
Worth looking at both the quoted text here and •especially• the linked page, which is quite good.
I’ll add another item of my own. The first screenshot mentions giving an LLM the task of “implementing an HTTP server in JavaScript from scratch” in 90 minutes. Sounds impressive, right? Until you remember that every open-source Javascript HTTP server in existence ••was in the training data••.
1/
@inthehands My company houses a number of pro-AI engineers. I asked them to evaluate Claude (which I understand is their favorite) by asking it to build something broken in a very specific, novel way.
My suggestion wasn't taken seriously, unfortunately, but I suppose it might be interesting. -
IT WAS IN THE TRAINING DATA. Your test gave the machine a pile of correct answers and free license to plagiarize.
I remember people being wowed that Claude Code could implement a complete C compiler. But somehow it doesn’t sound quite as impressive when you phrase it as “given every existing C compiler as input, the LLM can produce a C compiler as output.”
2/
@inthehands
Grep could do it faster.
-
@inthehands (Completely aside: I sometimes feel like I ought to learn French *just* to be able to read that one in its original language. It is very special to me; it was the first book I ever read myself.
Even more completely aside: Someone ought to give Hayao Miyazaki the rights to create an animated interpretation of it, and supply him with however much coffee and pure adrenochrome needed to keep him alive, active and in good spirits for however long that takes. And reward him with whatever he wanted - for example, the exclusive right to hunt Sam Altman for sport.)
@datarama @inthehands Miyazaki's headed for retirement (again), may I propose Guillermo del Toro?
-
RE: https://unstable.systems/@jneen/116618931097778342
Worth looking at both the quoted text here and •especially• the linked page, which is quite good.
I’ll add another item of my own. The first screenshot mentions giving an LLM the task of “implementing an HTTP server in JavaScript from scratch” in 90 minutes. Sounds impressive, right? Until you remember that every open-source Javascript HTTP server in existence ••was in the training data••.
1/
@inthehands Additionally, why would I want a tool to write me an http server? I already have lots of http servers.
What's next? Are you going to invent a tool that will mail me thousands of free AOL CD's?
-
The reasons we consider plagiarism to be cheating in school and malpractice in professional contexts are many of the same reasons that LLMs are not going to replace all knowledge-based human labor. (Details left as an exercise for the reader.)
And yes, a whole lot of what LLMs do •would• count as plagiarism if one of my students did it manually, and •should• count as plagiarism just the same if they use a machine to do it — not just in a “that’s cheating!!” sense, but more importantly in a “that’s not really doing the work” sense.
5/
One last example:
The first LLM code example that really made my eyes pop was early after the release of GPT, when somebody got it to combine Breakout with Conway’s Game of Life (a truly delightful idea). It worked!
Funny thing: the Breakout code and the Life code had a •completely• different style and flavor. Red flag. In about 15 minutes of web searching, I was able to find one of the projects (can’t remember if it was the Breakout or the Life half) which it had copied wholesale, with just a few variable renames. And the other half? It was in Python, but it used dictionaries where it really should have used objects — tons of `thing["prop"]` where it should have said `thing.prop`, and lots of other un-Pythonic stuff besides. It was a machine translate of code from another language, very likely Javascript.
The entire thing was a plagiarized Breakout and a plagiarized Game of Life, one transpiled, and all stuck together in a single run loop. To be fair, figuring out how to (1) run both halves of the logic from a single loop and (2) count the Life cells as Breakout bricks is work I'd cheer on from a second-semester intro CS student! It's not, however, quite what's being sold by these companies.
6/
-
IT WAS IN THE TRAINING DATA. Your test gave the machine a pile of correct answers and free license to plagiarize.
I remember people being wowed that Claude Code could implement a complete C compiler. But somehow it doesn’t sound quite as impressive when you phrase it as “given every existing C compiler as input, the LLM can produce a C compiler as output.”
2/
@inthehands @rayckeith Same with Mythos - the benchmark tests are in the training data. Cybergym tracks their performance and their website has a link to their github which refers to all the evaluation data being in huggingface (with the git repo link to it). It stands to reason that every model has been trained on the very thing they are then evaluated on.
-
@datarama @inthehands Miyazaki's headed for retirement (again), may I propose Guillermo del Toro?
@Linza @inthehands Miyazaki was born to make that movie, and it is a cosmic injustice of the highest order if he doesn't get to do it.
-
@Lily_and_frog @inthehands I think that has been said for plumbing—$1 for the part, $45 for the five minutes it took to install the part, $400 for the years of experience it takes to know which part and how to install it.
Yep!
But in the case of LLMs, it's thousands and thousands of years of experience + hundreds and hundreds of years of just... labelling the source material! -
@inthehands (Completely aside: I sometimes feel like I ought to learn French *just* to be able to read that one in its original language. It is very special to me; it was the first book I ever read myself.
Even more completely aside: Someone ought to give Hayao Miyazaki the rights to create an animated interpretation of it, and supply him with however much coffee and pure adrenochrome needed to keep him alive, active and in good spirits for however long that takes. And reward him with whatever he wanted - for example, the exclusive right to hunt Sam Altman for sport.)
@datarama @inthehands That is a reasonably achievable goal since it is pretty short and written in a very accessible language.
So feel encouraged to try if you wish to!
(As a French person, I never had to learn it the hard way, and admire anyone who does) -
IT WAS IN THE TRAINING DATA. Your test gave the machine a pile of correct answers and free license to plagiarize.
I remember people being wowed that Claude Code could implement a complete C compiler. But somehow it doesn’t sound quite as impressive when you phrase it as “given every existing C compiler as input, the LLM can produce a C compiler as output.”
2/
We need to consider the possibility that soon cp(1) will become sentient.
https://exple.tive.org/blarg/2026/02/12/the-pride-of-subject-hometown-here/
https://exple.tive.org/blarg/2026/02/07/on-the-crank-spectrum/
-
@datarama @inthehands That is a reasonably achievable goal since it is pretty short and written in a very accessible language.
So feel encouraged to try if you wish to!
(As a French person, I never had to learn it the hard way, and admire anyone who does)@temptoetiam @inthehands I can read Borges (slowly and with embarrassingly frequent dictionary breaks) in the original Spanish! That's actually one of the ways I maintained being able to at least read the language (though I struggle with understanding spoken Spanish, if it's spoken at a natural pace) since back when I took Spanish in high school.
(I can also read Russian children's literature - *very* far from my goal of being able to read the Strugatsky brothers' science fiction in the original language.
) -
One last example:
The first LLM code example that really made my eyes pop was early after the release of GPT, when somebody got it to combine Breakout with Conway’s Game of Life (a truly delightful idea). It worked!
Funny thing: the Breakout code and the Life code had a •completely• different style and flavor. Red flag. In about 15 minutes of web searching, I was able to find one of the projects (can’t remember if it was the Breakout or the Life half) which it had copied wholesale, with just a few variable renames. And the other half? It was in Python, but it used dictionaries where it really should have used objects — tons of `thing["prop"]` where it should have said `thing.prop`, and lots of other un-Pythonic stuff besides. It was a machine translate of code from another language, very likely Javascript.
The entire thing was a plagiarized Breakout and a plagiarized Game of Life, one transpiled, and all stuck together in a single run loop. To be fair, figuring out how to (1) run both halves of the logic from a single loop and (2) count the Life cells as Breakout bricks is work I'd cheer on from a second-semester intro CS student! It's not, however, quite what's being sold by these companies.
6/
As per my posts, I have the luxury of not having LLM vendors shoved down my throat, and I generally avoid them for ethical reasons:
https://hachyderm.io/@inthehands/116581463138461199
But because this all these questions about the usage and limits of these tools keep crashing through my doors, all of our doors, whatever we think of the ethical showstoppers, well…
…fight off amazing percentages of LLM overhype with this one weird question.
/end
-
One last example:
The first LLM code example that really made my eyes pop was early after the release of GPT, when somebody got it to combine Breakout with Conway’s Game of Life (a truly delightful idea). It worked!
Funny thing: the Breakout code and the Life code had a •completely• different style and flavor. Red flag. In about 15 minutes of web searching, I was able to find one of the projects (can’t remember if it was the Breakout or the Life half) which it had copied wholesale, with just a few variable renames. And the other half? It was in Python, but it used dictionaries where it really should have used objects — tons of `thing["prop"]` where it should have said `thing.prop`, and lots of other un-Pythonic stuff besides. It was a machine translate of code from another language, very likely Javascript.
The entire thing was a plagiarized Breakout and a plagiarized Game of Life, one transpiled, and all stuck together in a single run loop. To be fair, figuring out how to (1) run both halves of the logic from a single loop and (2) count the Life cells as Breakout bricks is work I'd cheer on from a second-semester intro CS student! It's not, however, quite what's being sold by these companies.
6/
@inthehands i've noticed a trend in anecdotes recently where people are finding it harder to trace their novel-seeming LLM outputs back to inputs. i wonder if this is a result of them atomizing their inputs more finely, or being "better" at swapping the tokens around to make output look original. (an AI bro might argue that at some point human creativity is doing the same thing…)
-
As per my posts, I have the luxury of not having LLM vendors shoved down my throat, and I generally avoid them for ethical reasons:
https://hachyderm.io/@inthehands/116581463138461199
But because this all these questions about the usage and limits of these tools keep crashing through my doors, all of our doors, whatever we think of the ethical showstoppers, well…
…fight off amazing percentages of LLM overhype with this one weird question.
/end
@inthehands Going further: if you could google the source for a HTTP server in JavaScript in 2015, the LLM should be able to output one in 0 (zero) minutes or it's failed.
-
@inthehands (Completely aside: I sometimes feel like I ought to learn French *just* to be able to read that one in its original language. It is very special to me; it was the first book I ever read myself.
Even more completely aside: Someone ought to give Hayao Miyazaki the rights to create an animated interpretation of it, and supply him with however much coffee and pure adrenochrome needed to keep him alive, active and in good spirits for however long that takes. And reward him with whatever he wanted - for example, the exclusive right to hunt Sam Altman for sport.)
@datarama
I know just enough French to have read Le Petit Prince in the original language (with some struggle), and…it really is beautiful in French in a way that translations don't capture. “On ne voit bien qu'avec le coeur” can translate into English quite directly as “One does not see well but with the heart,” but it just doesn't have the same poetry and magic at all. -
@temptoetiam @inthehands I can read Borges (slowly and with embarrassingly frequent dictionary breaks) in the original Spanish! That's actually one of the ways I maintained being able to at least read the language (though I struggle with understanding spoken Spanish, if it's spoken at a natural pace) since back when I took Spanish in high school.
(I can also read Russian children's literature - *very* far from my goal of being able to read the Strugatsky brothers' science fiction in the original language.
)@datarama @inthehands knowing Spanish is a great stepping stone to learn any other romance languages!
Bon courage à toi
