“I poisoned my own mind.”
-
@dalias so, another bold claim without any ground.
@dalias sorry mate, but you have no fucking clue what you're talking about.
-
@dalias so, another bold claim without any ground.
@bonkers No, it's not a "bold claim". There is no model of reality or how anything works in these systems, only pattern regurgitation in ways that correlate/align with context. If you get any comments that appear to explain motivations, they're copies of someone else's ideas from a similar context, not something reasoned to apply to your code.
The way you support or refute what I'm saying is through examination & analysis of the mechanisms, not examination of the output and whether it looks like what you would expect from human reasoning. "Looks like human reasoning" is the deception working as intended, and if you're trying to assess it from that standpoint, you're falling for the brain rotting program.
-
@bonkers No, it's not a "bold claim". There is no model of reality or how anything works in these systems, only pattern regurgitation in ways that correlate/align with context. If you get any comments that appear to explain motivations, they're copies of someone else's ideas from a similar context, not something reasoned to apply to your code.
The way you support or refute what I'm saying is through examination & analysis of the mechanisms, not examination of the output and whether it looks like what you would expect from human reasoning. "Looks like human reasoning" is the deception working as intended, and if you're trying to assess it from that standpoint, you're falling for the brain rotting program.
@dalias no, I'm just saying that it's a useful tool in an experienced engineer's hands. Everything else is your own assumption.
-
@bonkers The kind of comments in LLM codeslop are the opposite of useful. They do not document motivations why something was done the way it was, connections to external contracts, etc. They're imprecise restatement of what's in the code. The same bad types of comments novice human codemonkeys write when told to comment their code better.
@dalias @bonkers at my job, I work on the open source software of a very famous science project, so people often think they'll "help out" by pointing agents at our codebase. The LLM slop that well-meaning people have started foisting on me and my colleagues almost always has the same thing in common; it's almost always extraordinary and unnecessarily verbose. Pull request descriptions are long paragraphs of repeated information with bold section headers and emoji. Bits of logic that could be a single line are instead broken into multiple hyper-specific functions. function docstrings are useless reframings of exactly what the code says, without actually saying what situations the function should be used for. It's a mental burden that these people dump on us and say "you sort this out, I'm so helpful!" and I just want to reach through the screen and shake their shoulders and tell them that if they actually want to be helpful they can just read our contributing docs and write stuff themselves!
-
@dalias @bonkers at my job, I work on the open source software of a very famous science project, so people often think they'll "help out" by pointing agents at our codebase. The LLM slop that well-meaning people have started foisting on me and my colleagues almost always has the same thing in common; it's almost always extraordinary and unnecessarily verbose. Pull request descriptions are long paragraphs of repeated information with bold section headers and emoji. Bits of logic that could be a single line are instead broken into multiple hyper-specific functions. function docstrings are useless reframings of exactly what the code says, without actually saying what situations the function should be used for. It's a mental burden that these people dump on us and say "you sort this out, I'm so helpful!" and I just want to reach through the screen and shake their shoulders and tell them that if they actually want to be helpful they can just read our contributing docs and write stuff themselves!
@zrb @dalias true, and the only useful thing here would be rejecting such PRs without spending much time on them.
They could, theoretically, utilize the clanker to produce the change, but they should at least be able to understand and summarize it.
My point is, it's a tool. A hammer could also be used for building good stuff and for killing people.
-
@zrb @dalias true, and the only useful thing here would be rejecting such PRs without spending much time on them.
They could, theoretically, utilize the clanker to produce the change, but they should at least be able to understand and summarize it.
My point is, it's a tool. A hammer could also be used for building good stuff and for killing people.
-
@bonkers @dalias that was the first thing we did, and really all that does is weed out the people and automated agents that respect policies, and select for people and agents that don't. Companies already don't care about open source licenses, and don't respect
robots.txtwhen scraping, so it's no surprise they don't respect community policies either. -
@bonkers @dalias that was the first thing we did, and really all that does is weed out the people and automated agents that respect policies, and select for people and agents that don't. Companies already don't care about open source licenses, and don't respect
robots.txtwhen scraping, so it's no surprise they don't respect community policies either. -
-
@dalias no, I'm just saying that it's a useful tool in an experienced engineer's hands. Everything else is your own assumption.
There is no understanding or real logic involved in LLM output. Only dice rolls.
Those comments only say something that plausibly sounds like what the batch of text called a "comment" has been determined to be LIKELY to say.
That's not a guess, that's how the thing works. The LLM is the thing doing the guessing. It isn't capable of understanding or reasoning.
-
@dalias no, I'm just saying that it's a useful tool in an experienced engineer's hands. Everything else is your own assumption.
@bonkers i'm sure there's a tool involved
-
There is no understanding or real logic involved in LLM output. Only dice rolls.
Those comments only say something that plausibly sounds like what the batch of text called a "comment" has been determined to be LIKELY to say.
That's not a guess, that's how the thing works. The LLM is the thing doing the guessing. It isn't capable of understanding or reasoning.
@violetmadder @dalias you just shouldn't use it for things that need reasoning. It's pretty good at standard tasks, so keep it at that.
-
@violetmadder @dalias you just shouldn't use it for things that need reasoning. It's pretty good at standard tasks, so keep it at that.
Comments need reasoning. Code structure needs reasoning.
In those tasks, the plausible mimicry of what's needed is not just insufficient, but dangerous.
-
@violetmadder @dalias life is dangerous. The longer you live, the closer you are to your death.
Hammers and axes are dangerous, they can be used as a weapon.
We just have to admit that the clankers will stick around. We just have to minimize the danger.
-
@brib @glyph @davidgerard Uh, oh, excusing people because they're weak-minded or surrender to “convenience” is not the way to go, I believe.
@chroma @glyph @davidgerard I never used the words "weak-minded" or "convenience".
-
@violetmadder @dalias life is dangerous. The longer you live, the closer you are to your death.
Hammers and axes are dangerous, they can be used as a weapon.
We just have to admit that the clankers will stick around. We just have to minimize the danger.
Hammers and axes are primarily tools, first and foremost. That's what they are usually optimized for. An axe for battle is not built the same way as an axe for wood.
Machine learning is a massively powerful and potentially useful tool, sure-- but "AI" in its current form has been designed and implemented, intentionally, as a weapon. Hammers and axes generally have smooth handles, as a basic safety feature-- they don't dig vampiric spines into the hands that wield them. They don't PRETEND to chop wood or hammer nails while camouflaging jagged leftover pieces that are going to cause future problems. ...They don't read a fucking GitHub issue title and take it as a command to install an agent on your computer that starts doing who knows what other shit-- look, this analogy can't even BEGIN to touch what's really going on here.
Generative "AI" is a signal jammer, flooding the zone with an avalanche of bullshit that's poisoning the wells of information, junking up communications and news with crap, flooding software with unreviewable masses of code stuffed with time bomb errors at best if not malicious who knows what-- while the companies responsible encourage not just the devaluing of labor but de-skilling the whole workforce-- not to mention eroding critical thinking skills in general and trying to replace human interaction with their own social engineering. They're enclosing the digital commons, manipulating hardware prices to push home computing out of reach, stealing everyone's work, all while feeding surveillance infrastructure that would make the Nazis swoon, AND setting up the world economy for a devastating crash the likes of which we haven't seen since 1929.
But sure, let's act like it's just a cute little useful ball peen hammer that we just have to be careful not to bump our own thumbs with. Fucksakes.
-
T tanyakaroli@expressional.social shared this topic
