all the criticism has been said, all the takes been had.
-
@ainmosni
I also believe we're currently in an unsustainable bubble where the true cost of all of this inference is subsidized by people who have been allowed to accrue too much capital. And I feel like in fairly short order people are going to have to start reckoning with whether all this is worth it when venture capital isn't covering 90% of the real cost for them.@ainmosni I'll also admit that I find it very difficult to truly understand what people who have an emotional relationship with AI are going through. Because my reaction to a software program feigning any kind of emotional understanding or empathy is to be extremely repulsed.
-
@ainmosni I'll also admit that I find it very difficult to truly understand what people who have an emotional relationship with AI are going through. Because my reaction to a software program feigning any kind of emotional understanding or empathy is to be extremely repulsed.
@ainmosni And I do worry that the people who work for the AI companies seem to be imo somewhat delusional. Like asking a model what it wants to do in "retirement" and giving it a blog to express itself on rather than just turning it off. When they more than anyone should know it's just an algorithm and is therefore incapable of having feelings or opinions.
-
RE: https://hails.org/@hailey/116657391001259044
all the criticism has been said, all the takes been had. the only metaphor i have been finding consistently useful for understanding what is happening with people and "AI" is addiction, and specifically gambling addiction.
@jonny As well... there are a few people I was amazed to talk to, especially science fiction loving guys higher up at tech companies (who are like, everywhere) which have an almost animistic belief that GAI currently exists. Got called small-minded, etc, which is fine. Sources and research didn't phase them.
They sincerely believe in it, and you know you can't replace a belief with data and information. My anthropology courses have never been better applied than recently with the cultish behaviour around genAI. I'm interested in how it feels for people who get that gambling rush about genAI, and how it ties into beliefs, could it maybe strengthen the affect?
-
RE: https://hails.org/@hailey/116657391001259044
all the criticism has been said, all the takes been had. the only metaphor i have been finding consistently useful for understanding what is happening with people and "AI" is addiction, and specifically gambling addiction.
mandatory
"LLMs are slot-machines": https://doctorow.medium.com/https-pluralistic-net-2025-08-16-jackpot-salience-bias-2a696501bba7
and
-
RE: https://hails.org/@hailey/116657391001259044
all the criticism has been said, all the takes been had. the only metaphor i have been finding consistently useful for understanding what is happening with people and "AI" is addiction, and specifically gambling addiction.
@jonny I believe addiction treatment centres are seeing the same as you. Is that peer reviewed hard research or them capitalizing on what people feel like they need treatment for? I don't know.
https://www.naadac.org/treating-internet-addiction-ai-pornography-social-media-online-gambling-gaming -
RE: https://hails.org/@hailey/116657391001259044
all the criticism has been said, all the takes been had. the only metaphor i have been finding consistently useful for understanding what is happening with people and "AI" is addiction, and specifically gambling addiction.
@jonny Just one more commit. This time will be different.
-
@jonny exactly, when i talk to people who maybe just use ai a little bit, their reaponses quite align with things you’d hear from drug addict, some excuses, avoidance, bit of shame.
And i agree that the patern is finally similar to gaming/gambling addiction@prema @jonny "I can quit anytime, I just choose not to."
"I am only a social slopper. I never consume slop when alone at home."
"I only use chatbots with my morning coffee. Coffee doesn't taste good without a chatbot."
"Without a chatbot I wouldn't know what to do with my hands."
"I would have quit, but I do not want to get fat."
"All my colleagues use slop, I also need to use slop for the sake of networking."
"If I quit nobody is going to invite me out anymore because I won't be fun to be around." -
@ainmosni
You have to feel the rush to understand it@jonny @ainmosni exactly. winning early and being on the dopamine chasing side of nuerospicy helps kick start it.
i tried AI a bit and omg when its works SHOT TO THE VIEN. it takes the things you hate feeling and makes it gone but moving through bad feelings is like exercise you cant keep sick gains without daily effort
-
@jonny Yeah, I never got that rush, I think my brain is just too stuck in "cynical arsehole" mode for that to work on me.
@ainmosni @jonny I don't get the rush, but I *do* get sucked into loops I can't easily break out of (I have OCD). I get sucked into "completionist mode" in games, which means that if I play a game where there are collectibles you can buy with actual money, I'm quite likely to make very stupid decisions.
I've learned my lesson, I avoid such games scrupulously, and I steer clear of gambling.
And I am very distraught that my workplace now wants me to use AI.
-
i love gambling. i have used "AI" extensively. it feels the same.
Agreed. I don’t gamble because I know I’m vulnerable. And the use of Ai really is the just one more sensation. It’s too damned easy to get a plausible output. And it’s hard to stop.
-
@concretedog @jonny @ainmosni also don't gamble but I think I am closer to the avoidance due to not wanting to fall into a hole than the complete lack of interest, I internally predict election results and other stuff just don't put actual money on it.
Don't smoke because I know it would be hard to stop and rarely play computer games for the same reason. Don't use LLMs and a big part of that is seeing other people get sucked in.
-
RE: https://hails.org/@hailey/116657391001259044
all the criticism has been said, all the takes been had. the only metaphor i have been finding consistently useful for understanding what is happening with people and "AI" is addiction, and specifically gambling addiction.
@jonny so this thread could be a case study
-
@jonny so this thread could be a case study
@glassresistor
Yeah I saw that and got sad -
i love gambling. i have used "AI" extensively. it feels the same.
So, look. One shot rewriting the whole test suite in another language is probably not great to do, but what happened here is so much worse than you are expecting.
https://github.com/RsyncProject/rsync/pull/903/
This does not "translate tests into pytest" or a unit testing framework, it writes its own testing framework where tests are whole python scripts that redefine basic test functions in every script. Surely there would be a single way to "run rsync and get the results" - nope, well, there is, but then every test file will randomly redefine its own
_run_and_capturefunction. So like now rsync needs a test suite for its test suite.If instead of telling an LLM to "rewrite the tests in python" you just searched "python testing" you would find the pytest docs. And then you would find examples. And then you could write fixtures to deduplicate all the prior shell script setup and teardown stuff, and so on. But since it was just "rewrite the tests in python" its now worse than before, and the odds of the rewrite actually being a 100% faithful translation are close to 0.
-
So, look. One shot rewriting the whole test suite in another language is probably not great to do, but what happened here is so much worse than you are expecting.
https://github.com/RsyncProject/rsync/pull/903/
This does not "translate tests into pytest" or a unit testing framework, it writes its own testing framework where tests are whole python scripts that redefine basic test functions in every script. Surely there would be a single way to "run rsync and get the results" - nope, well, there is, but then every test file will randomly redefine its own
_run_and_capturefunction. So like now rsync needs a test suite for its test suite.If instead of telling an LLM to "rewrite the tests in python" you just searched "python testing" you would find the pytest docs. And then you would find examples. And then you could write fixtures to deduplicate all the prior shell script setup and teardown stuff, and so on. But since it was just "rewrite the tests in python" its now worse than before, and the odds of the rewrite actually being a 100% faithful translation are close to 0.
@jonny *screaming intensifies*
-
So, look. One shot rewriting the whole test suite in another language is probably not great to do, but what happened here is so much worse than you are expecting.
https://github.com/RsyncProject/rsync/pull/903/
This does not "translate tests into pytest" or a unit testing framework, it writes its own testing framework where tests are whole python scripts that redefine basic test functions in every script. Surely there would be a single way to "run rsync and get the results" - nope, well, there is, but then every test file will randomly redefine its own
_run_and_capturefunction. So like now rsync needs a test suite for its test suite.If instead of telling an LLM to "rewrite the tests in python" you just searched "python testing" you would find the pytest docs. And then you would find examples. And then you could write fixtures to deduplicate all the prior shell script setup and teardown stuff, and so on. But since it was just "rewrite the tests in python" its now worse than before, and the odds of the rewrite actually being a 100% faithful translation are close to 0.
-
So, look. One shot rewriting the whole test suite in another language is probably not great to do, but what happened here is so much worse than you are expecting.
https://github.com/RsyncProject/rsync/pull/903/
This does not "translate tests into pytest" or a unit testing framework, it writes its own testing framework where tests are whole python scripts that redefine basic test functions in every script. Surely there would be a single way to "run rsync and get the results" - nope, well, there is, but then every test file will randomly redefine its own
_run_and_capturefunction. So like now rsync needs a test suite for its test suite.If instead of telling an LLM to "rewrite the tests in python" you just searched "python testing" you would find the pytest docs. And then you would find examples. And then you could write fixtures to deduplicate all the prior shell script setup and teardown stuff, and so on. But since it was just "rewrite the tests in python" its now worse than before, and the odds of the rewrite actually being a 100% faithful translation are close to 0.
I think the modal situation here is that the people are reading none or very little of what is being generated by the LLM, so the tests have a special role: Tests function as the pull arm on the slot machine, you just generate until tests pass, and that's a jackpot. Obviously that's meaningless when the tests are meaningless, so tests take on a very different meaning and role in slot machine coding.
Previously we would write careful test conditions that were based off some real problem or an understanding of what the code under test did, and had a specific thing they were intended to protect against. Tests move slow and are designed to protect us against the things we know can go wrong. When we learn of a new wrong thing, we add a test.
LLM tests have the form of tests but don't do the same thing. They often test nothing, and are just expressions of truisms that the probabilistic text space explored while generating. They have strongly worded names but end up actually asserting that basic language features work as expected. Because it is not us writing tests for ourselves, where we only harm ourselves by making them weak, they function instead as a passively obfuscated justification for the code that the LLM generates. The user wants the tests to pass. The LLM provides.
The tests are theater: they are the play field for the slot machine. They are mild, surmountable, need to fail a few times to be plausible, but must eventually pass within the expected generation loop window to deliver the payout.
-
So, look. One shot rewriting the whole test suite in another language is probably not great to do, but what happened here is so much worse than you are expecting.
https://github.com/RsyncProject/rsync/pull/903/
This does not "translate tests into pytest" or a unit testing framework, it writes its own testing framework where tests are whole python scripts that redefine basic test functions in every script. Surely there would be a single way to "run rsync and get the results" - nope, well, there is, but then every test file will randomly redefine its own
_run_and_capturefunction. So like now rsync needs a test suite for its test suite.If instead of telling an LLM to "rewrite the tests in python" you just searched "python testing" you would find the pytest docs. And then you would find examples. And then you could write fixtures to deduplicate all the prior shell script setup and teardown stuff, and so on. But since it was just "rewrite the tests in python" its now worse than before, and the odds of the rewrite actually being a 100% faithful translation are close to 0.
@jonny@neuromatch.social oh holy FUCK
it is so bad -
I think the modal situation here is that the people are reading none or very little of what is being generated by the LLM, so the tests have a special role: Tests function as the pull arm on the slot machine, you just generate until tests pass, and that's a jackpot. Obviously that's meaningless when the tests are meaningless, so tests take on a very different meaning and role in slot machine coding.
Previously we would write careful test conditions that were based off some real problem or an understanding of what the code under test did, and had a specific thing they were intended to protect against. Tests move slow and are designed to protect us against the things we know can go wrong. When we learn of a new wrong thing, we add a test.
LLM tests have the form of tests but don't do the same thing. They often test nothing, and are just expressions of truisms that the probabilistic text space explored while generating. They have strongly worded names but end up actually asserting that basic language features work as expected. Because it is not us writing tests for ourselves, where we only harm ourselves by making them weak, they function instead as a passively obfuscated justification for the code that the LLM generates. The user wants the tests to pass. The LLM provides.
The tests are theater: they are the play field for the slot machine. They are mild, surmountable, need to fail a few times to be plausible, but must eventually pass within the expected generation loop window to deliver the payout.
@jonny “tests have a special role”
️ -
I think the modal situation here is that the people are reading none or very little of what is being generated by the LLM, so the tests have a special role: Tests function as the pull arm on the slot machine, you just generate until tests pass, and that's a jackpot. Obviously that's meaningless when the tests are meaningless, so tests take on a very different meaning and role in slot machine coding.
Previously we would write careful test conditions that were based off some real problem or an understanding of what the code under test did, and had a specific thing they were intended to protect against. Tests move slow and are designed to protect us against the things we know can go wrong. When we learn of a new wrong thing, we add a test.
LLM tests have the form of tests but don't do the same thing. They often test nothing, and are just expressions of truisms that the probabilistic text space explored while generating. They have strongly worded names but end up actually asserting that basic language features work as expected. Because it is not us writing tests for ourselves, where we only harm ourselves by making them weak, they function instead as a passively obfuscated justification for the code that the LLM generates. The user wants the tests to pass. The LLM provides.
The tests are theater: they are the play field for the slot machine. They are mild, surmountable, need to fail a few times to be plausible, but must eventually pass within the expected generation loop window to deliver the payout.
Here's an example from some code that was thrust at me this week. The rest of the tests try a bit harder to look like tests, but this one is perplexing.
What does it test? The function name suggests its a smoke test. LLMs love to call things smoke tests. That would suggest this would be an early-run test that fails loudly if some basic precondition - like having ffmpeg - fails. Or, I guess we are smoke testing the
ensure_ffmpegfunction? Anyway who knows. However we first check if ffmpeg or ffprobe are present, which is exactly whatensure_ffmpegdoes. If they aren't present, a warning tells us that ffmpeg/ffprobe are required for the video tests, which makes it seem like this should be a parameterizing test that controls which tests are run, which of course it does not do.So the test literally does nothing and cannot possibly fail, but says it does at least two things, because to an LLM something saying it does something is the same thing as it actually doing that thing.
