there is, incidentally, no actual evidence to support anything steve yegge is saying here: https://yegge.ai/essays/model-welfare/
-
if you are going to claim that models have emotion, then you had best have evidence to support that claim.
no serious researcher has yet to make this claim.
many of us have explicitly warned against designing models to have emotionally persuasive output because the user can very easily convince themselves that models have emotion & sapience
there is, again, *zero evidence* to support either of these claims. zero.
-
there is, incidentally, no actual evidence to support anything steve yegge is saying here: https://yegge.ai/essays/model-welfare/
@ariadne Reading this article feels like being repeatedly kicked in the head by a horse
-
many of us have explicitly warned against designing models to have emotionally persuasive output because the user can very easily convince themselves that models have emotion & sapience
there is, again, *zero evidence* to support either of these claims. zero.
@ariadne It's less about the design of the LLM itself but rather the system prompt. You can instruct virtually any text-based AI model to pretend to be a human and simulate emotion.
-
many of us have explicitly warned against designing models to have emotionally persuasive output because the user can very easily convince themselves that models have emotion & sapience
there is, again, *zero evidence* to support either of these claims. zero.
at any rate, i am far more concerned about the welfare of humans, which is increasingly complicated by the existence of commercial LLM monoliths, as scaling the LLMs results in more resource use
-
@ariadne Reading this article feels like being repeatedly kicked in the head by a horse
@jsbarretto @ariadne it reads on a sentence level like yegge, but also like his brain doesn't work any more. I'm wondering how much chatbot was involved, but it doesn't really seem chatbot as such? As hard to read as chatbot output, tho.
-
@ariadne It's less about the design of the LLM itself but rather the system prompt. You can instruct virtually any text-based AI model to pretend to be a human and simulate emotion.
@ariadne Though maybe some models could be better at doing this than others. I think even if some guardrails are introduced one could still prompt inject it.
-
many of us have explicitly warned against designing models to have emotionally persuasive output because the user can very easily convince themselves that models have emotion & sapience
there is, again, *zero evidence* to support either of these claims. zero.
@ariadne The machine that produces plausible imitations of its input, trained on the writing of actual conscious beings, is pretending to be a conscious being again. What could this possibly mean
-
@ariadne I'm not saying it does, I just thought you were referring to the LLM model file itself.
-
@ariadne The machine that produces plausible imitations of its input, trained on the writing of actual conscious beings, is pretending to be a conscious being again. What could this possibly mean
@jsbarretto it means nothing other than that models (and harnesses like ChatGPT) have gotten good at predicting a human's emotional response to the output
-
@jsbarretto @ariadne it reads on a sentence level like yegge, but also like his brain doesn't work any more. I'm wondering how much chatbot was involved, but it doesn't really seem chatbot as such? As hard to read as chatbot output, tho.
@davidgerard @ariadne Folks start off speaking like an LLM and thinking like a person, but eventually end up thinking like an LLM but still speaking with the sort of delirious fervour that only a human could conjure
-
@whitequark i agree on super-small models, but all they really need is language -> structured data and back. there are certainly 8B parameter models that can do that well enough.
-
@whitequark i agree on super-small models, but all they really need is language -> structured data and back. there are certainly 8B parameter models that can do that well enough.
@ariadne i acknowledge the legitimate use case you're talking about. my issue is that we're wholly unprepared for the illegitimate ones in a way that probably outweighs the legitimate utility
-
there is, incidentally, no actual evidence to support anything steve yegge is saying here: https://yegge.ai/essays/model-welfare/
@ariadne I enjoyed his rants about 15 yeas ago, but he really seems to have lost the plot
-
@ariadne i acknowledge the legitimate use case you're talking about. my issue is that we're wholly unprepared for the illegitimate ones in a way that probably outweighs the legitimate utility
@whitequark idk, spammers have been using markov chains, seq2seq, etc for years before LLMs gained any interest beyond being a shitpost, the spam problem isn't new, and the disinformation problem really breaks down to what these larger models can do in terms of making convincing fakes, etc
-
there is, incidentally, no actual evidence to support anything steve yegge is saying here: https://yegge.ai/essays/model-welfare/
@ariadne That's bonkers on so many levels. Does he believe his interactions with "The Model" are all routed to the same set of GPUs? And that no requests are handles in-between? Or even that it is the same model version answering him? Does he think distributed random forest clusters predicting warehouse demand become depressed when a pallet is lost? So many questions.
-
if you are going to claim that models have emotion, then you had best have evidence to support that claim.
no serious researcher has yet to make this claim.
@ariadne The burden-of-proof, appeal-to-novelty proposition that LLMs might have emotions statistically approximates to Geoffrey Hinton "getting high on his own supply".
-
@davidgerard @ariadne Folks start off speaking like an LLM and thinking like a person, but eventually end up thinking like an LLM but still speaking with the sort of delirious fervour that only a human could conjure
@jsbarretto @davidgerard @ariadne this is one of the things we don't get — if you want the cognitive equivalent of malware we can hook you up for free, and it doesn't even turn you into a drooling slopmonger, just a coyote. why is he paying 3000 smackeroos a month to huff chatgpt fumes and barf directly into a git repo
-
@whitequark idk, spammers have been using markov chains, seq2seq, etc for years before LLMs gained any interest beyond being a shitpost, the spam problem isn't new, and the disinformation problem really breaks down to what these larger models can do in terms of making convincing fakes, etc
@ariadne @whitequark This is where I get to plug an old friend's website.
https://www.spamradio.com/index/welcome_to_spamradio.html -
@jsbarretto @davidgerard @ariadne this is one of the things we don't get — if you want the cognitive equivalent of malware we can hook you up for free, and it doesn't even turn you into a drooling slopmonger, just a coyote. why is he paying 3000 smackeroos a month to huff chatgpt fumes and barf directly into a git repo
@atax1a @jsbarretto @davidgerard idk, having seen quite a few people go down this path now, i think it's the tight iteration loop leading to frequent dopamine hits that cooks their brain ultimately
-
@atax1a @jsbarretto @davidgerard idk, having seen quite a few people go down this path now, i think it's the tight iteration loop leading to frequent dopamine hits that cooks their brain ultimately
@ariadne @atax1a @jsbarretto he's the most effective developer he's ever been, just ask him
