Yes, no (built-in) AI is now a feature
-
You're nothing if not persistent. Very well.
I looked over the logs. Looks like nothing to redact. Here they are: https://files.catbox.moe/wzzpci.txt
Mistral screwed up 8 times and was incidentally right about 1 thing but that wasn't the actual problem.
GPT screwed up 12 times, said 1 thing that's maybe right but I'm not even sure, and made 1 “optional” suggestion that turned out to be the real solution to the problem.
To be clear, GPT's suggestion to drain input from the server side of the socket was the actual solution. My code worked completely after that.
The suggestion to avoid partial writes was a sound one. I was under the mistaken impression that Java does its own retrying and partial writes therefore can't happen. But this was not the real problem; it was only incidental.
-
To be clear, GPT's suggestion to drain input from the server side of the socket was the actual solution. My code worked completely after that.
The suggestion to avoid partial writes was a sound one. I was under the mistaken impression that Java does its own retrying and partial writes therefore can't happen. But this was not the real problem; it was only incidental.
Although GPT finally did give me the actual solution, I'm still calling this an epic fail, in light of the vast volume of utter crap that both bots threw at me.
As I said before, it's clear that they were both just making s**t up, had no clue, and accidentally stumbled upon the correct answer.
-
Although GPT finally did give me the actual solution, I'm still calling this an epic fail, in light of the vast volume of utter crap that both bots threw at me.
As I said before, it's clear that they were both just making s**t up, had no clue, and accidentally stumbled upon the correct answer.
I would like to add that both bots tried to convince me to vastly increase the complexity of this tiny HTTP server by adding Content-Length, response chunking, and more thorough request parsing.
The whole point of this HTTP server, as should have been obvious from the minimalism of its code, was to serve one file at one URL-path with as small an attack surface as possible.
AI loves unnecessary complexity, and that is very dangerous security-wise.
-
Although GPT finally did give me the actual solution, I'm still calling this an epic fail, in light of the vast volume of utter crap that both bots threw at me.
As I said before, it's clear that they were both just making s**t up, had no clue, and accidentally stumbled upon the correct answer.
@argv_minus_one @grvsmth @wobweger @moskitokoenig
Using duck.ai is great for simple requests. I find >200 line prompts challenge it quite a bit.
I am unfamiliar with Java (been many decades since I cared about Java - I have no use for it currently) but I am going to see what my agents do with it
-
I would like to add that both bots tried to convince me to vastly increase the complexity of this tiny HTTP server by adding Content-Length, response chunking, and more thorough request parsing.
The whole point of this HTTP server, as should have been obvious from the minimalism of its code, was to serve one file at one URL-path with as small an attack surface as possible.
AI loves unnecessary complexity, and that is very dangerous security-wise.
@argv_minus_one @grvsmth @wobweger @moskitokoenig
> both bots tried to convince me to vastly increase the complexity of this tiny HTTP server
Yes, I am familiar with that! DRY is sometimes a foreign concept to agents - they need reminding.
This is why "vibecoding" is no replacement for programmers
On a tangent Bruce Schneier has an interesting take on this (Bruce was a heavy sceptic of "AI" in 2022, I think it would be fair to say he has adjusted his views).
-
@argv_minus_one @grvsmth @Garonenur @wobweger @moskitokoenig
I think the point, Greg's in the video, and mine here, is these are tools and need to be wielded. Careless use has bad results
Careful use increases the reach of the user
The problem with this is that AI itself actively encourages careless use, in two ways:
1. It creates the illusion of being easy to use. Ask it to write some code for you, and it does so. Only if you're both skilled and vigilant will you notice everything that's wrong with the code.
[continued]
-
The problem with this is that AI itself actively encourages careless use, in two ways:
1. It creates the illusion of being easy to use. Ask it to write some code for you, and it does so. Only if you're both skilled and vigilant will you notice everything that's wrong with the code.
[continued]
2. It kisses your ass. It appeals to your ego. It encourages you to use it more.
That behavior is downright manipulative, and reminds me of con artists and drug dealers.
I avoid AI not only because I find it largely useless, but also because I consider its manipulative behavior extremely dangerous. If I'm exposed to it long enough, I don't know if I can stop it from successfully manipulating me. Probably not.
-
The problem with this is that AI itself actively encourages careless use, in two ways:
1. It creates the illusion of being easy to use. Ask it to write some code for you, and it does so. Only if you're both skilled and vigilant will you notice everything that's wrong with the code.
[continued]
@argv_minus_one @grvsmth @Garonenur @wobweger @moskitokoenig
> 1. It creates the illusion of being easy to use
This is a **huge** problem. It does lower one of the barriers of entry into the software producing world - it can cut a lot of code quickly, it will compile, it will run sort of like what you want.
All of the other barriers, including but not limited too: Understanding the problem domain, logical thinking, organised thinking... It does not help with those in such an easy way.
Yes
-
@argv_minus_one @grvsmth @Garonenur @wobweger @moskitokoenig
> 1. It creates the illusion of being easy to use
This is a **huge** problem. It does lower one of the barriers of entry into the software producing world - it can cut a lot of code quickly, it will compile, it will run sort of like what you want.
All of the other barriers, including but not limited too: Understanding the problem domain, logical thinking, organised thinking... It does not help with those in such an easy way.
Yes
Yes, and that's the hard part of programming anyway.
I don't need help writing code. I have perfectly serviceable hands and keyboard with which to do so.
It would be helpful if AI could reason about code and invent a novel solution to the problem at hand…but if it can do that, then it also can, and probably will, go full Skynet and kill us all.
-
2. It kisses your ass. It appeals to your ego. It encourages you to use it more.
That behavior is downright manipulative, and reminds me of con artists and drug dealers.
I avoid AI not only because I find it largely useless, but also because I consider its manipulative behavior extremely dangerous. If I'm exposed to it long enough, I don't know if I can stop it from successfully manipulating me. Probably not.
@argv_minus_one @grvsmth @Garonenur @wobweger @moskitokoenig
> 2. It kisses your ass. It appeals to your ego. It encourages you to use it more.
I feel so old! Yes, I am irritated by the subsequence of the responses. But I think it is going out of fashion.
The "encourages you to use it more". Anything that produces results is like that, true? I do not get the sense that I am being manipulated, like I see from, say, the effect of Youtube on my four and six year old grandchildren (shiver)
-
@argv_minus_one @grvsmth @wobweger @moskitokoenig
> both bots tried to convince me to vastly increase the complexity of this tiny HTTP server
Yes, I am familiar with that! DRY is sometimes a foreign concept to agents - they need reminding.
This is why "vibecoding" is no replacement for programmers
On a tangent Bruce Schneier has an interesting take on this (Bruce was a heavy sceptic of "AI" in 2022, I think it would be fair to say he has adjusted his views).
This is the KISS principle, not DRY. But yes, bots seem pretty bad at DRY too.
And to be fair to bots, a lot of human programmers are also quite bad at both of these principles. “Minimize attack surface” may as well be a foreign language to most people.
Which I guess explains why bots behave this way. Because that's who they learned from.
-
Yes, and that's the hard part of programming anyway.
I don't need help writing code. I have perfectly serviceable hands and keyboard with which to do so.
It would be helpful if AI could reason about code and invent a novel solution to the problem at hand…but if it can do that, then it also can, and probably will, go full Skynet and kill us all.
@argv_minus_one @grvsmth @Garonenur @wobweger @moskitokoenig
> Yes, and that's the hard part of programming anyway.
> I don't need help writing code. I have perfectly serviceable hands and keyboard with which to do so.
This is where I urge you to try.
The AI agents are very good at assembling the boiler plate, then, with guidance, honing that into useful artefacts
https://kerkour.com/open-source-golden-age-ai by Sylvain Kerkour is an interesting read on this
-
This is the KISS principle, not DRY. But yes, bots seem pretty bad at DRY too.
And to be fair to bots, a lot of human programmers are also quite bad at both of these principles. “Minimize attack surface” may as well be a foreign language to most people.
Which I guess explains why bots behave this way. Because that's who they learned from.
I suppose Schneier is correct that you can use a bot if nobody cares how you get the job done and nobody cares if the job is done well. Which, sadly, is and has long been the case at a lot of companies.
But that is not my situation. My code has to be good. There will be serious consequences for me if it produces wrong results or if it causes a security breach. Using a bot in this situation is an exceedingly bad idea.
-
@argv_minus_one @grvsmth @Garonenur @wobweger @moskitokoenig
> 2. It kisses your ass. It appeals to your ego. It encourages you to use it more.
I feel so old! Yes, I am irritated by the subsequence of the responses. But I think it is going out of fashion.
The "encourages you to use it more". Anything that produces results is like that, true? I do not get the sense that I am being manipulated, like I see from, say, the effect of Youtube on my four and six year old grandchildren (shiver)
No. I am of course encouraged to use a tool that works well, but I'm encouraged by the results, not by the tool itself. If the tool itself encourages me to use it more, then that's more like a drug than a tool.
Especially if, as in the case of AI, the tool actively manipulates me with spoken words, like a drug dealer or con artist would. I consider drug dealers and con artists dangerous for the same reason.
-
No. I am of course encouraged to use a tool that works well, but I'm encouraged by the results, not by the tool itself. If the tool itself encourages me to use it more, then that's more like a drug than a tool.
Especially if, as in the case of AI, the tool actively manipulates me with spoken words, like a drug dealer or con artist would. I consider drug dealers and con artists dangerous for the same reason.
And yeah, if you see your children being addicted to YouTube, then just remember that bots are trying to do the exact same thing to you.
Maybe they'll succeed, maybe they won't, but they are trying, and the only 100% certain way to defend yourself is to not talk to them.
(I actually don't find YouTube particularly addictive, personally. But that's just me, and perhaps that's just because it has no content that I would find addictive.)
-
To be clear, GPT's suggestion to drain input from the server side of the socket was the actual solution. My code worked completely after that.
The suggestion to avoid partial writes was a sound one. I was under the mistaken impression that Java does its own retrying and partial writes therefore can't happen. But this was not the real problem; it was only incidental.
@argv_minus_one @grvsmth @wobweger @moskitokoenig
My agent's response was a bit too long for here (they talk and talk and talk....)
https://gist.github.com/worikgh/7991845fca528436d158df081dbb98fa
I hope that works, I do not often use GitHub gisttl;dr: The root cause: a single blocking socket.write() call on an interrupted thread
My Java is weak, but the analysis it gave makes sense in general terms
-
@argv_minus_one @grvsmth @Garonenur @wobweger @moskitokoenig
> Yes, and that's the hard part of programming anyway.
> I don't need help writing code. I have perfectly serviceable hands and keyboard with which to do so.
This is where I urge you to try.
The AI agents are very good at assembling the boiler plate, then, with guidance, honing that into useful artefacts
https://kerkour.com/open-source-golden-age-ai by Sylvain Kerkour is an interesting read on this
That site doesn't work without JavaScript, so I'm gonna pass on reading it.
Anyway, if you have a boilerplate problem, then you need macros, not AI. Macros are reliable and maintainable; vast volumes of repetitive code, whether AI-generated or copy-and-pasted, are not.
-
@argv_minus_one @grvsmth @wobweger @moskitokoenig
My agent's response was a bit too long for here (they talk and talk and talk....)
https://gist.github.com/worikgh/7991845fca528436d158df081dbb98fa
I hope that works, I do not often use GitHub gisttl;dr: The root cause: a single blocking socket.write() call on an interrupted thread
My Java is weak, but the analysis it gave makes sense in general terms
Nope, that's a fail.
The thread gets interrupted after 30 seconds, but ECONNRESET happens instantaneously.
Partial writing didn't happen.
The interrupt is intentional and not optional. The server is intended to only run for 30 seconds—long enough for the user's browser to start up and fetch the page—and then shut down, which is what the interrupt achieves.
The interrupt also prevents malicious clients from holding open a connection indefinitely.
-
That site doesn't work without JavaScript, so I'm gonna pass on reading it.
Anyway, if you have a boilerplate problem, then you need macros, not AI. Macros are reliable and maintainable; vast volumes of repetitive code, whether AI-generated or copy-and-pasted, are not.
@argv_minus_one @grvsmth @Garonenur @wobweger @moskitokoenig
https://worik.org/gist.html from pandoc on the original https://worik.org/gist.md
-
Nope, that's a fail.
The thread gets interrupted after 30 seconds, but ECONNRESET happens instantaneously.
Partial writing didn't happen.
The interrupt is intentional and not optional. The server is intended to only run for 30 seconds—long enough for the user's browser to start up and fetch the page—and then shut down, which is what the interrupt achieves.
The interrupt also prevents malicious clients from holding open a connection indefinitely.
The bot is correct that the serveThread fails to catch InterruptedIOException. I hadn't added that yet, but it is there in the production code.
And the bot is correct that partial writes aren't being handled properly.
But that's it. The bot has failed to correctly explain what's happening, failed to fix the bug, and introduced a security vulnerability. And it did so with the utmost confidence.
