Yes, no (built-in) AI is now a feature
-
Well, I just looked over the lengthy transcripts of the chats I had with two different bots on the topic (Mistral and GPT-5.4-mini, as it turns out), and I *was* going to go over them and redact them…
…but then it occurred to me that you're probably just going to have your bot “summarize” the conversations anyway…
…in which case this is a waste of my time, so I'm not going to bother.
I have, however, gone ahead and saved the transcripts, for my own future amusement and/or to bring receipts to someone who's actually willing to read them. So thanks for reminding me to do that.
-
I have, however, gone ahead and saved the transcripts, for my own future amusement and/or to bring receipts to someone who's actually willing to read them. So thanks for reminding me to do that.
@argv_minus_one @grvsmth @wobweger @moskitokoenig
I am glad you did not do the work correcting the transcripts, that is not what interests me.
What interests me is the question. From memory (interface here will not let me rep.y and look at thread simultaneously easilly) the question was a computing problem, you got ten answers and four were correct.
I am interested. I
-
Your being astounded does not mean LLMs have surpassed humans, only that they've surpassed you.
Anyway, Greg K-H is talking about using LLMs to find bugs, like a static analyzer. That's grand and all, but you're talking about vibe code, and that's not what he's talking about.
@argv_minus_one @grvsmth @Garonenur @wobweger @moskitokoenig
> Greg K-H is talking about using LLMs to find bugs,
More than that, he urges people to develop patches, using LLM prompts, so the bug report comes with a solution (unless I my recall is fault.ty, I watched said video yesterday but only once)
His point is that doing so exposes that the LLM generates false positives, and propper, simple, use of LLMs can reduce them
-
@argv_minus_one @grvsmth @Garonenur @wobweger @moskitokoenig
> Greg K-H is talking about using LLMs to find bugs,
More than that, he urges people to develop patches, using LLM prompts, so the bug report comes with a solution (unless I my recall is fault.ty, I watched said video yesterday but only once)
His point is that doing so exposes that the LLM generates false positives, and propper, simple, use of LLMs can reduce them
@argv_minus_one @grvsmth @Garonenur @wobweger @moskitokoenig
I think the point, Greg's in the video, and mine here, is these are tools and need to be wielded. Careless use has bad results
Careful use increases the reach of the user
-
@argv_minus_one @grvsmth @wobweger @moskitokoenig
I am glad you did not do the work correcting the transcripts, that is not what interests me.
What interests me is the question. From memory (interface here will not let me rep.y and look at thread simultaneously easilly) the question was a computing problem, you got ten answers and four were correct.
I am interested. I
You're nothing if not persistent. Very well.
I looked over the logs. Looks like nothing to redact. Here they are: https://files.catbox.moe/wzzpci.txt
Mistral screwed up 8 times and was incidentally right about 1 thing but that wasn't the actual problem.
GPT screwed up 12 times, said 1 thing that's maybe right but I'm not even sure, and made 1 “optional” suggestion that turned out to be the real solution to the problem.
-
You're nothing if not persistent. Very well.
I looked over the logs. Looks like nothing to redact. Here they are: https://files.catbox.moe/wzzpci.txt
Mistral screwed up 8 times and was incidentally right about 1 thing but that wasn't the actual problem.
GPT screwed up 12 times, said 1 thing that's maybe right but I'm not even sure, and made 1 “optional” suggestion that turned out to be the real solution to the problem.
To be clear, GPT's suggestion to drain input from the server side of the socket was the actual solution. My code worked completely after that.
The suggestion to avoid partial writes was a sound one. I was under the mistaken impression that Java does its own retrying and partial writes therefore can't happen. But this was not the real problem; it was only incidental.
-
To be clear, GPT's suggestion to drain input from the server side of the socket was the actual solution. My code worked completely after that.
The suggestion to avoid partial writes was a sound one. I was under the mistaken impression that Java does its own retrying and partial writes therefore can't happen. But this was not the real problem; it was only incidental.
Although GPT finally did give me the actual solution, I'm still calling this an epic fail, in light of the vast volume of utter crap that both bots threw at me.
As I said before, it's clear that they were both just making s**t up, had no clue, and accidentally stumbled upon the correct answer.
-
Although GPT finally did give me the actual solution, I'm still calling this an epic fail, in light of the vast volume of utter crap that both bots threw at me.
As I said before, it's clear that they were both just making s**t up, had no clue, and accidentally stumbled upon the correct answer.
I would like to add that both bots tried to convince me to vastly increase the complexity of this tiny HTTP server by adding Content-Length, response chunking, and more thorough request parsing.
The whole point of this HTTP server, as should have been obvious from the minimalism of its code, was to serve one file at one URL-path with as small an attack surface as possible.
AI loves unnecessary complexity, and that is very dangerous security-wise.
-
Although GPT finally did give me the actual solution, I'm still calling this an epic fail, in light of the vast volume of utter crap that both bots threw at me.
As I said before, it's clear that they were both just making s**t up, had no clue, and accidentally stumbled upon the correct answer.
@argv_minus_one @grvsmth @wobweger @moskitokoenig
Using duck.ai is great for simple requests. I find >200 line prompts challenge it quite a bit.
I am unfamiliar with Java (been many decades since I cared about Java - I have no use for it currently) but I am going to see what my agents do with it
-
I would like to add that both bots tried to convince me to vastly increase the complexity of this tiny HTTP server by adding Content-Length, response chunking, and more thorough request parsing.
The whole point of this HTTP server, as should have been obvious from the minimalism of its code, was to serve one file at one URL-path with as small an attack surface as possible.
AI loves unnecessary complexity, and that is very dangerous security-wise.
@argv_minus_one @grvsmth @wobweger @moskitokoenig
> both bots tried to convince me to vastly increase the complexity of this tiny HTTP server
Yes, I am familiar with that! DRY is sometimes a foreign concept to agents - they need reminding.
This is why "vibecoding" is no replacement for programmers
On a tangent Bruce Schneier has an interesting take on this (Bruce was a heavy sceptic of "AI" in 2022, I think it would be fair to say he has adjusted his views).
-
@argv_minus_one @grvsmth @Garonenur @wobweger @moskitokoenig
I think the point, Greg's in the video, and mine here, is these are tools and need to be wielded. Careless use has bad results
Careful use increases the reach of the user
The problem with this is that AI itself actively encourages careless use, in two ways:
1. It creates the illusion of being easy to use. Ask it to write some code for you, and it does so. Only if you're both skilled and vigilant will you notice everything that's wrong with the code.
[continued]
-
The problem with this is that AI itself actively encourages careless use, in two ways:
1. It creates the illusion of being easy to use. Ask it to write some code for you, and it does so. Only if you're both skilled and vigilant will you notice everything that's wrong with the code.
[continued]
2. It kisses your ass. It appeals to your ego. It encourages you to use it more.
That behavior is downright manipulative, and reminds me of con artists and drug dealers.
I avoid AI not only because I find it largely useless, but also because I consider its manipulative behavior extremely dangerous. If I'm exposed to it long enough, I don't know if I can stop it from successfully manipulating me. Probably not.
-
The problem with this is that AI itself actively encourages careless use, in two ways:
1. It creates the illusion of being easy to use. Ask it to write some code for you, and it does so. Only if you're both skilled and vigilant will you notice everything that's wrong with the code.
[continued]
@argv_minus_one @grvsmth @Garonenur @wobweger @moskitokoenig
> 1. It creates the illusion of being easy to use
This is a **huge** problem. It does lower one of the barriers of entry into the software producing world - it can cut a lot of code quickly, it will compile, it will run sort of like what you want.
All of the other barriers, including but not limited too: Understanding the problem domain, logical thinking, organised thinking... It does not help with those in such an easy way.
Yes
-
@argv_minus_one @grvsmth @Garonenur @wobweger @moskitokoenig
> 1. It creates the illusion of being easy to use
This is a **huge** problem. It does lower one of the barriers of entry into the software producing world - it can cut a lot of code quickly, it will compile, it will run sort of like what you want.
All of the other barriers, including but not limited too: Understanding the problem domain, logical thinking, organised thinking... It does not help with those in such an easy way.
Yes
Yes, and that's the hard part of programming anyway.
I don't need help writing code. I have perfectly serviceable hands and keyboard with which to do so.
It would be helpful if AI could reason about code and invent a novel solution to the problem at hand…but if it can do that, then it also can, and probably will, go full Skynet and kill us all.
-
2. It kisses your ass. It appeals to your ego. It encourages you to use it more.
That behavior is downright manipulative, and reminds me of con artists and drug dealers.
I avoid AI not only because I find it largely useless, but also because I consider its manipulative behavior extremely dangerous. If I'm exposed to it long enough, I don't know if I can stop it from successfully manipulating me. Probably not.
@argv_minus_one @grvsmth @Garonenur @wobweger @moskitokoenig
> 2. It kisses your ass. It appeals to your ego. It encourages you to use it more.
I feel so old! Yes, I am irritated by the subsequence of the responses. But I think it is going out of fashion.
The "encourages you to use it more". Anything that produces results is like that, true? I do not get the sense that I am being manipulated, like I see from, say, the effect of Youtube on my four and six year old grandchildren (shiver)
-
@argv_minus_one @grvsmth @wobweger @moskitokoenig
> both bots tried to convince me to vastly increase the complexity of this tiny HTTP server
Yes, I am familiar with that! DRY is sometimes a foreign concept to agents - they need reminding.
This is why "vibecoding" is no replacement for programmers
On a tangent Bruce Schneier has an interesting take on this (Bruce was a heavy sceptic of "AI" in 2022, I think it would be fair to say he has adjusted his views).
This is the KISS principle, not DRY. But yes, bots seem pretty bad at DRY too.
And to be fair to bots, a lot of human programmers are also quite bad at both of these principles. “Minimize attack surface” may as well be a foreign language to most people.
Which I guess explains why bots behave this way. Because that's who they learned from.
-
Yes, and that's the hard part of programming anyway.
I don't need help writing code. I have perfectly serviceable hands and keyboard with which to do so.
It would be helpful if AI could reason about code and invent a novel solution to the problem at hand…but if it can do that, then it also can, and probably will, go full Skynet and kill us all.
@argv_minus_one @grvsmth @Garonenur @wobweger @moskitokoenig
> Yes, and that's the hard part of programming anyway.
> I don't need help writing code. I have perfectly serviceable hands and keyboard with which to do so.
This is where I urge you to try.
The AI agents are very good at assembling the boiler plate, then, with guidance, honing that into useful artefacts
https://kerkour.com/open-source-golden-age-ai by Sylvain Kerkour is an interesting read on this
-
This is the KISS principle, not DRY. But yes, bots seem pretty bad at DRY too.
And to be fair to bots, a lot of human programmers are also quite bad at both of these principles. “Minimize attack surface” may as well be a foreign language to most people.
Which I guess explains why bots behave this way. Because that's who they learned from.
I suppose Schneier is correct that you can use a bot if nobody cares how you get the job done and nobody cares if the job is done well. Which, sadly, is and has long been the case at a lot of companies.
But that is not my situation. My code has to be good. There will be serious consequences for me if it produces wrong results or if it causes a security breach. Using a bot in this situation is an exceedingly bad idea.
-
@argv_minus_one @grvsmth @Garonenur @wobweger @moskitokoenig
> 2. It kisses your ass. It appeals to your ego. It encourages you to use it more.
I feel so old! Yes, I am irritated by the subsequence of the responses. But I think it is going out of fashion.
The "encourages you to use it more". Anything that produces results is like that, true? I do not get the sense that I am being manipulated, like I see from, say, the effect of Youtube on my four and six year old grandchildren (shiver)
No. I am of course encouraged to use a tool that works well, but I'm encouraged by the results, not by the tool itself. If the tool itself encourages me to use it more, then that's more like a drug than a tool.
Especially if, as in the case of AI, the tool actively manipulates me with spoken words, like a drug dealer or con artist would. I consider drug dealers and con artists dangerous for the same reason.
-
No. I am of course encouraged to use a tool that works well, but I'm encouraged by the results, not by the tool itself. If the tool itself encourages me to use it more, then that's more like a drug than a tool.
Especially if, as in the case of AI, the tool actively manipulates me with spoken words, like a drug dealer or con artist would. I consider drug dealers and con artists dangerous for the same reason.
And yeah, if you see your children being addicted to YouTube, then just remember that bots are trying to do the exact same thing to you.
Maybe they'll succeed, maybe they won't, but they are trying, and the only 100% certain way to defend yourself is to not talk to them.
(I actually don't find YouTube particularly addictive, personally. But that's just me, and perhaps that's just because it has no content that I would find addictive.)
