I keep seeing lots of people saying "LLMs are like compilers/assemblers for prompts"

[rdviii@famichiki.jp]

@cwebber mostly agree, especially about them not being compilers, but some compilers aren't deterministic. You'll get a different result in memory layout or optimization sometimes. Especially for quantum compilers, where the compilation process itself is known to be NP hard, so heuristics are used.

[kkarhan@infosec.space]

@eramdam @cwebber +1

[nobody@mastodon.acm.org]

@cwebber
PGO go brrrrr

[baloouriza@social.tulsa.ok.us]

@cwebber This is more like the Pentium 4 idea of predictive branching, but with even larger pipeline stalls. Except the P4 could still do math.

[ryanc@infosec.exchange]

@drwho @cstanhope @mcc @mntmn @cwebber Honestly, I would prefer LLM generated code over grad student generated code.

[yaleman@mastodon.social]

@rdviii Ok but who's actually talking about *quantum compilers* when they are just saying "compilers" as a general term? ... other than people who work exclusively on QC's, who would be ... an incredibly tiny minority

[kkarhan@infosec.space]

@cwebber precisely that!

A #shitposting - Program is anything but #reproduceable and I want #ReproduceableBuilds for #auditability, #security and #transparency.

• That's the whole reason I do @OS1337: To have something so fundamentally simple and compact that it is (at least in theory - at some point) financially feasible to crowdfund complete code audits of the entire system.
  • I don't want people to trust me blindly, but to earn trust in the few things I code.

That's why I treat any "#AI" / #AIslop the same way @dolphin treat any leaks from Nintendo:

• I'm not even gonna look at it!

[pautasso@scholar.social]

@cwebber if, just like with asm, reading and reviewing generated code is not longer a necessary thing, then the productivity bottleneck shifts to how much time is spent "engineering" the prompt.

[eviloatmeal@ak.angelstrapped.com]

@cwebber @joeyh If someone invented an LLM that gave me powerups and metaprogression, I might be slightly interested.

[cwebber@social.coop]

@ansuz @joeyh And of course there is the question, what is and isn't a compiler? Aren't all functions compilers?

Indeed, Blender's rendering system is in many ways a compiler for images.

But we don't use that way, because it's not helpful, even though Blender and ffmpeg are MORE of compilers than LLMs are. People are reaching for "LLMs might be compilers!" because of the thing they want it to *do* rather than how it *acts*, even though Blender and ffmpeg are by far, under those definitions, much more of compilers than LLMs are.

[cwebber@social.coop]

@ansuz @joeyh To put it another way: even though we could call Blender and ffmpeg compilers in a way that would be hard to argue with, we don't, and it wouldn't be useful if we did because we wouldn't understand each other well.

Please don't call LLMs compilers.

[hackbod@mastodon.social]

@ansuz @joeyh @cwebber

Ah but even if you can use a specific seed and try to use this to call it a "compiler", your compiler here is the very specific sets of weights within that model, and any change breaks its determinism. I think there being one and exactly one possible implementation to get the specified set of outputs can count as an actual compiler.

[fiore@brain.worm.pink]

@eramdam@erambert.me @cwebber@social.coop Twitter tech influencers have been saying this for years already

[thomasjwebb@mastodon.social]

@cwebber It's pretty simple. If it's like a compiler, then why do you check in the output? And with all the work put into making compilers more efficient (not just making the *output* more efficient), why does it take so long and require an internet connection?

[thomasjwebb@mastodon.social]

@ansuz @cwebber @joeyh the reproducibility will also get pulled out as the model you used gets sunset. Unless all you check in is a series of prompts and a bunch of tests and simply assume future models will do a better job.

It could even be a problem where future generations want a "vintage AI" look for whatever reason and unlike so many past generations of tech, they simply won't be able to because it was a cloud service and the company is long gone.

[elrohir@mastodon.gal]

@cwebber I'm only going to say that if natural human language was suitable for expressing expected response results in a predictable and well defined manner, we wouldn't have spent the last 50 years memorizing rulebooks that say "MUST means that the definition is an absolute requirement of the specification."

At this point my rage almost goes beyond whether it's a LLM or a Witch's Cauldron taking the prompts. I want to scream at people NATURAL LANGUAGE IS NOT A RECOMMENDABLE INPUT FORMAT.

[millihertz@oldbytes.space]

@cwebber well, it was until C99 anyway...

[fogti@chaos.social]

@natty @cwebber Java2K (wait, that's more like stochastic interpreter)

[ansuz@gts.cryptography.dog]

@thomasjwebb @cwebber @joeyh ​

Local models like llama could be reworked to accept a seed for their RNG. There'd be less risk of them becoming unavailable, and they'd be both deterministic and reproducible, but they'd still be terrible for all the other reasons that LLMs are terrible .

"Sovereign" and reproducible slop is still just slop

[deech@mastodon.social]

@cwebber I think we can compromise and call them really shitty compilers.