wall_e@ioc.exchange
Indlæg
-
I propose declaring October 21 to henceforth be International Static Website Day. -
OK!@ssilvonen @jonny that's why prompt injection/context pollution attacks are so powerful.
No matter how detailed your English language instructions, write enough of them and they'll start to contradict each other depending on context.
That's how you can get an LLM to produce output it has been instructed over- and over again not to produce, by inventing a context under which the instructions you input still align with it's priming.
And perhaps unintuitively, larger (ie. better?) models are more prone to this than smaller because they seem to be more "suggestible" (excuse the anthropomorphism)
-
OK!@ssilvonen @jonny yes the verbose explanation are the markdown files that are being ingested as context by the agent.
The "you" is the LLM.
And of course almost none of it is actually written by humans. LLMs all the way down.
-
OK!@jonny I just can't with the whole concept of this 🫠
10k lines of "code", which is actually just plain text English prose (which sure as hell won't contain any contradictions), to get the non-deterministic inference machine to behave somewhat predictably and mimic an actual engineered software product.Surely there can't be any way to do this more efficiently
