Although trained in physics, I worked in the computing industry with pride and purpose for over 40 years.
-
@nickzoic since you're a CAD person, I'll share a story I shared with him during this discussion.
My dad was a product engineer and product designer, and specialised in autocad and pro-engineer. I always thought he was smart but never really understood what he did. Then, one day I went in to his office on a weekend (to play games on one of the PCs, we didn't have one at home)...@nickzoic dad always used to ask me questions about how I thought things worked (to keep me curious I guess). I caught a glimpse of him working over his shoulder and asked him what he was doing, and in the process I asked him how he make a model of a chair.
He opened an empty document and proceeded to build a proper sculpted, curved, chair (the style with baton for a back rest). No child's sketch. A real chair. He did it in about 5 minutes... -
@nickzoic dad always used to ask me questions about how I thought things worked (to keep me curious I guess). I caught a glimpse of him working over his shoulder and asked him what he was doing, and in the process I asked him how he make a model of a chair.
He opened an empty document and proceeded to build a proper sculpted, curved, chair (the style with baton for a back rest). No child's sketch. A real chair. He did it in about 5 minutes...@nickzoic ...and he did it in ways I don't think 99% of the population would even imagine, with simple geometry, rotations, and extrusions and symmetry, and extruded swept shapes. (what I'd later know as CSG).
It completely blew my mind that there was this entirely different way of seeing the world.
And that's the joy of learning, and I really cannot fathom why people would want to skip that. -
@tmcfarlane to me that sounds like baseline neurotypical human. Minimize energy expenditure at any cost.
If something doesn't explode immediately: it works.
If something doesn't work: fiddle with it minimally until it does."Good enough", move on.
Spend that energy on something that makes you feel better about yourself, i.e. something that improves your perceived position in the hierarchy.
@robpike@einalex @robpike this particular friend is definitely not "neurotypical", and wouldn't have got as far as he has without being curious. I get the desire to just get stuff done.
In fact, he *did* describe what he'd genuinely learned about approaching tasks using an LLM, and that was in itself interesting.
But he hadn't learned how to do the thing he wanted to do. He'd learned how to pay someone else to do it for him, and he was weirdly excited about that bit. -
Although trained in physics, I worked in the computing industry with pride and purpose for over 40 years. And now I can do nothing but sit back and watch it destroy itself for no valid reason beyond hubris (if I'm being charitable).
Ineffable sadness watching something I once loved deliberately lose its soul.
@robpike sad
I'm a bit jealous that you were able to retire before the madness -
@robpike have you ever had a sense of what a precise artist you are? You are the apotheosis of beautiful creation. The reality is much of the industry doesn't need that beauty. I don't know how to reconcile that but it is at least consistent.
@nelson (blush) What a lovely compliment, thank you.
-
@einalex @robpike this particular friend is definitely not "neurotypical", and wouldn't have got as far as he has without being curious. I get the desire to just get stuff done.
In fact, he *did* describe what he'd genuinely learned about approaching tasks using an LLM, and that was in itself interesting.
But he hadn't learned how to do the thing he wanted to do. He'd learned how to pay someone else to do it for him, and he was weirdly excited about that bit.@tmcfarlane hrm, maybe he got gaslighted into "llms are really working"... in that case deep diving how they work would probably drastically change his outlook. @robpike
-
@tmcfarlane hrm, maybe he got gaslighted into "llms are really working"... in that case deep diving how they work would probably drastically change his outlook. @robpike
@einalex @robpike that's not reality. I was at his house for dinner, there were 6 of us there. I was the only one not using these tools. Everyone else is using them daily. Two of those were kids.
He built an actual object, a working case. It doesn't matter to him that it is built on the stolen work of others. He wanted to do a thing, and it did it (the object exists, that's not something that can be argued against). -
@einalex @robpike that's not reality. I was at his house for dinner, there were 6 of us there. I was the only one not using these tools. Everyone else is using them daily. Two of those were kids.
He built an actual object, a working case. It doesn't matter to him that it is built on the stolen work of others. He wanted to do a thing, and it did it (the object exists, that's not something that can be argued against).@tmcfarlane Nobody argues that llms don't produce something. The argument is that the thing created is a weird aproximation of the thing desired, and is produced at greater cost (which is differently distributed).
If you allow me an example of catastrophic failure of human engineering: bridge designs ignoring resonance.
- Tacoma bridge (wind caused)
- Millennium bridge (pedestrian caused)Both seemed like bridges before failure. Design without understanding made them traps instead.
-
@tmcfarlane Nobody argues that llms don't produce something. The argument is that the thing created is a weird aproximation of the thing desired, and is produced at greater cost (which is differently distributed).
If you allow me an example of catastrophic failure of human engineering: bridge designs ignoring resonance.
- Tacoma bridge (wind caused)
- Millennium bridge (pedestrian caused)Both seemed like bridges before failure. Design without understanding made them traps instead.
@tmcfarlane
Point being: something is something because of its characteristics, but is classified as something by human observers because of (usually) different, perceived features.We're comically bad at this, hence why the rules of trade in engineering are written in the blood of the victims of our prior misconceptions and mistakes.
"llm works" is a result of low requirements, rare/short use (hasn't failed yet), low stakes, and ignorance", rather than one of problem complexity.
-
Although trained in physics, I worked in the computing industry with pride and purpose for over 40 years. And now I can do nothing but sit back and watch it destroy itself for no valid reason beyond hubris (if I'm being charitable).
Ineffable sadness watching something I once loved deliberately lose its soul.
We could just call it "sabotage".
-
@tmcfarlane Nobody argues that llms don't produce something. The argument is that the thing created is a weird aproximation of the thing desired, and is produced at greater cost (which is differently distributed).
If you allow me an example of catastrophic failure of human engineering: bridge designs ignoring resonance.
- Tacoma bridge (wind caused)
- Millennium bridge (pedestrian caused)Both seemed like bridges before failure. Design without understanding made them traps instead.
@einalex @robpike On the specific cases of both Tacoma bridge, and the Millenium bridge, you do a disservice to the *engineers* that built those to the understood specs. Tacoma wasn't built with unprecedented winds in mind, and the millenium bridge was fine until unexpectedly large crowds did something we didn't realise they did.
We're doing a terrible thing with LLMs which is *exactly* the opposite of what the studious, learned, and hard working engineers on both those bridge actually did. -
@robpike I had a conversation with a close friend (also works in IT, we met 30 years ago).
He'd built a 3d print of a computer case using Claude to generate OpenSCAD.
What was most curious was his complete indifference to the generated code. He said he didn't learn any openscad syntax, and could not modify it without Claude if he wanted to.
The complete lack of curiosity involved was fascinating, and mildly chilling. Fine for a 48 year old man /maybe/, but is that what we want?Now that's one of my biggest concerns about it. Lots of loss of creativity; such a focus on the ends, with little regard to the means. And if you don't care for the means, which takes you to the end, you'll eventually be hindered trying to reach the end, too. #AI
CC: @robpike@hachyderm.io
-
@tmcfarlane
Point being: something is something because of its characteristics, but is classified as something by human observers because of (usually) different, perceived features.We're comically bad at this, hence why the rules of trade in engineering are written in the blood of the victims of our prior misconceptions and mistakes.
"llm works" is a result of low requirements, rare/short use (hasn't failed yet), low stakes, and ignorance", rather than one of problem complexity.
@einalex @robpike my friends mini-ATX motherboard is not sitting in a mini-ATX case that is definitely a case.
Now, he could have taken measurements, and built that case. Even modified another design. he could have understood what he did, and he would then be able to modify it without paying billionaires to do so.
LLMs are an intellectual catastrophe, and a moral failing. But we look stupid if we deny the daily reality of people using them. -
Now that's one of my biggest concerns about it. Lots of loss of creativity; such a focus on the ends, with little regard to the means. And if you don't care for the means, which takes you to the end, you'll eventually be hindered trying to reach the end, too. #AI
CC: @robpike@hachyderm.io@golemwire @robpike exactly. These tools skip the "why" and the "how" of the things they do. Even if you ask them to explain the "why" of something, you can't trust the answer. At least if I ask a good teacher why they want me to do X a certain way they can (I hope), give me an honest answer that I can research further (even if it is that best of honest answers "I don't know").
-
@einalex @robpike On the specific cases of both Tacoma bridge, and the Millenium bridge, you do a disservice to the *engineers* that built those to the understood specs. Tacoma wasn't built with unprecedented winds in mind, and the millenium bridge was fine until unexpectedly large crowds did something we didn't realise they did.
We're doing a terrible thing with LLMs which is *exactly* the opposite of what the studious, learned, and hard working engineers on both those bridge actually did.@tmcfarlane I'm not blaming them. I'm well aware they were working according to standards. I'm saying our understanding of the conditions was insufficient, so best practice engineering by lots of talented, experienced, well-intended humans still resulted in failure.
Contrast that with no-understanding low-effort llm usage. The notion that the results will "work" is comical and tragic.
-
@tmcfarlane I'm not blaming them. I'm well aware they were working according to standards. I'm saying our understanding of the conditions was insufficient, so best practice engineering by lots of talented, experienced, well-intended humans still resulted in failure.
Contrast that with no-understanding low-effort llm usage. The notion that the results will "work" is comical and tragic.
@einalex @robpike well they work right up until they don't. We had engineering to deal with that before, but apparently that was dull.
Would an AI Bro want to fly a plane his chat bot built? Or walk his family across a bridge one of them sketched up? (a few took their families down in a sub that an A-level mechanics student could have told you was a bad idea, so I'm not actually sure the answer to either of those is a "no"). -
@petrillic @robpike The recent Timothy Snyder essay on "Superpower Suicide" was posted/re-posted a lot this weekend. "Empires have risen and failed before, but to my knowledge no state has ever chosen to kill its own power, and succeeded with such rapidity." I hate to be susceptible to trying to make everything-look-like-everything-else but these two rots feel related if not foundationally then at least spiritually.
From this essay:
#Trump announced his main weapon would be tariffs, but then lost his trade war with China ...
It would serve the USA for Ukraine to win; but Trump switched to support Russia ...
Ukraine has performed ever better. The US failed to see its own interests ...
Trump left Iran with uranium in the hands of a more radical regime which holds new economic power ...
Instruments to influence Iranian society were willfully demolished in early 2026.
-
@robpike Ergo, capitalism tends to become ever-more detached from reality, and, if let alone, will destroy itself eventually. That which better comports with reality will remain, and will serve as foundations for whatever comes after. (Hopefully this includes us as a species, let alone high technology, but I suppose nothing is guaranteed.
-
@einalex @robpike well they work right up until they don't. We had engineering to deal with that before, but apparently that was dull.
Would an AI Bro want to fly a plane his chat bot built? Or walk his family across a bridge one of them sketched up? (a few took their families down in a sub that an A-level mechanics student could have told you was a bad idea, so I'm not actually sure the answer to either of those is a "no").@tmcfarlane not sure either..
To me, this is not working...that would be analog to dating a romance scammer and calling the experience "being in a relationship"
@robpike -
A forest burns but it comes back ever stronger.
Forests come back because thousands of individual seeds decide to sprout. Wounds heal because millions of cells take action.
Individuals need to reject LLMs, and the harmful companies and ideology behind them, if we're to see anything get better.
CC: @robpike@hachyderm.io
