Yes, no (built-in) AI is now a feature
-
I have, on several occasions.
One that stands out is when I asked a bot (I think Claude Haiku) a question about a programming topic I'm not entirely familiar with. It gave me 10 answers, 4 of which were obviously wrong based on the self-contained test case I presented it with, and 4 more of which immediately proved wrong upon cursory testing.
It was obvious that the bot was just making s**t up and had no idea what it was talking about.
2 of the answers were correct, so at least it wasn't a complete waste of my time.
But if I got 2 out of 10 questions wrong on a test at school, I wouldn't have been proclaimed a star pupil that everyone else has to keep up with. I'd have gotten an F, and then I'd have gotten a stern talking-to by my parents.
-
I have, on several occasions.
One that stands out is when I asked a bot (I think Claude Haiku) a question about a programming topic I'm not entirely familiar with. It gave me 10 answers, 4 of which were obviously wrong based on the self-contained test case I presented it with, and 4 more of which immediately proved wrong upon cursory testing.
It was obvious that the bot was just making s**t up and had no idea what it was talking about.
@argv_minus_one @grvsmth @wobweger @libreoffice @moskitokoenig
That is the exact opposite of my experience
You sure that you are not making it up?
-
@Garonenur @argv_minus_one @wobweger @libreoffice @grvsmth @moskitokoenig
Your comment about their output being shit is two years out of date
These days the code they produce is often very good.
Times are changing, and the development since 2022 has been astounding
The last nine months alone has seen huge advances.
More like two days. I frequently get sent slop by my coworkers.
No, it isn't. Not even remotely.
Yes, they are…in that we are speeding into an apocalypse of automated incompetence and consequent project collapse. I now fear deeply for the future, because of people like you.
The last nine months have been catastrophic.
-
@argv_minus_one @grvsmth @Garonenur @wobweger @moskitokoenig
Argumentum ad hominem ignored
I asserted that working on code is what (one thing) LLMs are good for.
I have been watching closely as LLMs and the harnesses have developed and they have astounded me.
There is little useful theory. That "text completion" could do all this is very surprising and I know of no theory to explain it
https://youtu.be/_MwMLPmMccs?si=RSTP2TcZCqmNGxX- is a short talk by Greg Kroah-Hartman you may find helpful.
Your being astounded does not mean LLMs have surpassed humans, only that they've surpassed you.
Anyway, Greg K-H is talking about using LLMs to find bugs, like a static analyzer. That's grand and all, but you're talking about vibe code, and that's not what he's talking about.
-
I have, on several occasions.
One that stands out is when I asked a bot (I think Claude Haiku) a question about a programming topic I'm not entirely familiar with. It gave me 10 answers, 4 of which were obviously wrong based on the self-contained test case I presented it with, and 4 more of which immediately proved wrong upon cursory testing.
It was obvious that the bot was just making s**t up and had no idea what it was talking about.
@argv_minus_one @grvsmth @wobweger @libreoffice @moskitokoenig
> One that stands out is when I asked a bot (I think Claude Haiku) a question about a programming topic I'm not entirely familiar with. It gave me 10 answers, 4 of which were obviously wrong based on the self-contained test case I presented it
I am curious.
May I have the question? I can try it out here
-
Your being astounded does not mean LLMs have surpassed humans, only that they've surpassed you.
Anyway, Greg K-H is talking about using LLMs to find bugs, like a static analyzer. That's grand and all, but you're talking about vibe code, and that's not what he's talking about.
I'm also not impressed with the high false-positive rate and psychologically-manipulative behavior Greg describes having experienced in his interactions with bots. Maybe he finds that tolerable, but I don't.
-
@Garonenur @argv_minus_one @wobweger @libreoffice @grvsmth @moskitokoenig
Your comment about their output being shit is two years out of date
These days the code they produce is often very good.
Times are changing, and the development since 2022 has been astounding
The last nine months alone has seen huge advances.
@worik @argv_minus_one @wobweger @libreoffice @grvsmth @moskitokoenig I have to read and review the code right today.
And it sucks.
It reuses old fashioned paradigms, because the old shit is more prevalent in the training data.
AND the users of those LLMs all start to become lazy and just believe what hallucinations the machine dishes them.
Code quality was terrible before, it will only go down now. -
@argv_minus_one @grvsmth @wobweger @libreoffice @moskitokoenig
That is the exact opposite of my experience
You sure that you are not making it up?
Well, I just looked over the lengthy transcripts of the chats I had with two different bots on the topic (Mistral and GPT-5.4-mini, as it turns out), and I *was* going to go over them and redact them…
…but then it occurred to me that you're probably just going to have your bot “summarize” the conversations anyway…
…in which case this is a waste of my time, so I'm not going to bother.
-
Well, I just looked over the lengthy transcripts of the chats I had with two different bots on the topic (Mistral and GPT-5.4-mini, as it turns out), and I *was* going to go over them and redact them…
…but then it occurred to me that you're probably just going to have your bot “summarize” the conversations anyway…
…in which case this is a waste of my time, so I'm not going to bother.
I have, however, gone ahead and saved the transcripts, for my own future amusement and/or to bring receipts to someone who's actually willing to read them. So thanks for reminding me to do that.
-
I have, however, gone ahead and saved the transcripts, for my own future amusement and/or to bring receipts to someone who's actually willing to read them. So thanks for reminding me to do that.
@argv_minus_one @grvsmth @wobweger @moskitokoenig
I am glad you did not do the work correcting the transcripts, that is not what interests me.
What interests me is the question. From memory (interface here will not let me rep.y and look at thread simultaneously easilly) the question was a computing problem, you got ten answers and four were correct.
I am interested. I
-
Your being astounded does not mean LLMs have surpassed humans, only that they've surpassed you.
Anyway, Greg K-H is talking about using LLMs to find bugs, like a static analyzer. That's grand and all, but you're talking about vibe code, and that's not what he's talking about.
@argv_minus_one @grvsmth @Garonenur @wobweger @moskitokoenig
> Greg K-H is talking about using LLMs to find bugs,
More than that, he urges people to develop patches, using LLM prompts, so the bug report comes with a solution (unless I my recall is fault.ty, I watched said video yesterday but only once)
His point is that doing so exposes that the LLM generates false positives, and propper, simple, use of LLMs can reduce them
-
@argv_minus_one @grvsmth @Garonenur @wobweger @moskitokoenig
> Greg K-H is talking about using LLMs to find bugs,
More than that, he urges people to develop patches, using LLM prompts, so the bug report comes with a solution (unless I my recall is fault.ty, I watched said video yesterday but only once)
His point is that doing so exposes that the LLM generates false positives, and propper, simple, use of LLMs can reduce them
@argv_minus_one @grvsmth @Garonenur @wobweger @moskitokoenig
I think the point, Greg's in the video, and mine here, is these are tools and need to be wielded. Careless use has bad results
Careful use increases the reach of the user
-
@argv_minus_one @grvsmth @wobweger @moskitokoenig
I am glad you did not do the work correcting the transcripts, that is not what interests me.
What interests me is the question. From memory (interface here will not let me rep.y and look at thread simultaneously easilly) the question was a computing problem, you got ten answers and four were correct.
I am interested. I
You're nothing if not persistent. Very well.
I looked over the logs. Looks like nothing to redact. Here they are: https://files.catbox.moe/wzzpci.txt
Mistral screwed up 8 times and was incidentally right about 1 thing but that wasn't the actual problem.
GPT screwed up 12 times, said 1 thing that's maybe right but I'm not even sure, and made 1 “optional” suggestion that turned out to be the real solution to the problem.
-
You're nothing if not persistent. Very well.
I looked over the logs. Looks like nothing to redact. Here they are: https://files.catbox.moe/wzzpci.txt
Mistral screwed up 8 times and was incidentally right about 1 thing but that wasn't the actual problem.
GPT screwed up 12 times, said 1 thing that's maybe right but I'm not even sure, and made 1 “optional” suggestion that turned out to be the real solution to the problem.
To be clear, GPT's suggestion to drain input from the server side of the socket was the actual solution. My code worked completely after that.
The suggestion to avoid partial writes was a sound one. I was under the mistaken impression that Java does its own retrying and partial writes therefore can't happen. But this was not the real problem; it was only incidental.
-
To be clear, GPT's suggestion to drain input from the server side of the socket was the actual solution. My code worked completely after that.
The suggestion to avoid partial writes was a sound one. I was under the mistaken impression that Java does its own retrying and partial writes therefore can't happen. But this was not the real problem; it was only incidental.
Although GPT finally did give me the actual solution, I'm still calling this an epic fail, in light of the vast volume of utter crap that both bots threw at me.
As I said before, it's clear that they were both just making s**t up, had no clue, and accidentally stumbled upon the correct answer.
-
Although GPT finally did give me the actual solution, I'm still calling this an epic fail, in light of the vast volume of utter crap that both bots threw at me.
As I said before, it's clear that they were both just making s**t up, had no clue, and accidentally stumbled upon the correct answer.
I would like to add that both bots tried to convince me to vastly increase the complexity of this tiny HTTP server by adding Content-Length, response chunking, and more thorough request parsing.
The whole point of this HTTP server, as should have been obvious from the minimalism of its code, was to serve one file at one URL-path with as small an attack surface as possible.
AI loves unnecessary complexity, and that is very dangerous security-wise.
-
Although GPT finally did give me the actual solution, I'm still calling this an epic fail, in light of the vast volume of utter crap that both bots threw at me.
As I said before, it's clear that they were both just making s**t up, had no clue, and accidentally stumbled upon the correct answer.
@argv_minus_one @grvsmth @wobweger @moskitokoenig
Using duck.ai is great for simple requests. I find >200 line prompts challenge it quite a bit.
I am unfamiliar with Java (been many decades since I cared about Java - I have no use for it currently) but I am going to see what my agents do with it
-
I would like to add that both bots tried to convince me to vastly increase the complexity of this tiny HTTP server by adding Content-Length, response chunking, and more thorough request parsing.
The whole point of this HTTP server, as should have been obvious from the minimalism of its code, was to serve one file at one URL-path with as small an attack surface as possible.
AI loves unnecessary complexity, and that is very dangerous security-wise.
@argv_minus_one @grvsmth @wobweger @moskitokoenig
> both bots tried to convince me to vastly increase the complexity of this tiny HTTP server
Yes, I am familiar with that! DRY is sometimes a foreign concept to agents - they need reminding.
This is why "vibecoding" is no replacement for programmers
On a tangent Bruce Schneier has an interesting take on this (Bruce was a heavy sceptic of "AI" in 2022, I think it would be fair to say he has adjusted his views).
-
@argv_minus_one @grvsmth @Garonenur @wobweger @moskitokoenig
I think the point, Greg's in the video, and mine here, is these are tools and need to be wielded. Careless use has bad results
Careful use increases the reach of the user
The problem with this is that AI itself actively encourages careless use, in two ways:
1. It creates the illusion of being easy to use. Ask it to write some code for you, and it does so. Only if you're both skilled and vigilant will you notice everything that's wrong with the code.
[continued]
-
The problem with this is that AI itself actively encourages careless use, in two ways:
1. It creates the illusion of being easy to use. Ask it to write some code for you, and it does so. Only if you're both skilled and vigilant will you notice everything that's wrong with the code.
[continued]
2. It kisses your ass. It appeals to your ego. It encourages you to use it more.
That behavior is downright manipulative, and reminds me of con artists and drug dealers.
I avoid AI not only because I find it largely useless, but also because I consider its manipulative behavior extremely dangerous. If I'm exposed to it long enough, I don't know if I can stop it from successfully manipulating me. Probably not.
