there is, incidentally, no actual evidence to support anything steve yegge is saying here: https://yegge.ai/essays/model-welfare/
-
@ariadne Reading this article feels like being repeatedly kicked in the head by a horse
@jsbarretto @ariadne it reads on a sentence level like yegge, but also like his brain doesn't work any more. I'm wondering how much chatbot was involved, but it doesn't really seem chatbot as such? As hard to read as chatbot output, tho.
-
@ariadne It's less about the design of the LLM itself but rather the system prompt. You can instruct virtually any text-based AI model to pretend to be a human and simulate emotion.
@ariadne Though maybe some models could be better at doing this than others. I think even if some guardrails are introduced one could still prompt inject it.
-
many of us have explicitly warned against designing models to have emotionally persuasive output because the user can very easily convince themselves that models have emotion & sapience
there is, again, *zero evidence* to support either of these claims. zero.
@ariadne The machine that produces plausible imitations of its input, trained on the writing of actual conscious beings, is pretending to be a conscious being again. What could this possibly mean
-
@ariadne I'm not saying it does, I just thought you were referring to the LLM model file itself.
-
@ariadne The machine that produces plausible imitations of its input, trained on the writing of actual conscious beings, is pretending to be a conscious being again. What could this possibly mean
@jsbarretto it means nothing other than that models (and harnesses like ChatGPT) have gotten good at predicting a human's emotional response to the output
-
@jsbarretto @ariadne it reads on a sentence level like yegge, but also like his brain doesn't work any more. I'm wondering how much chatbot was involved, but it doesn't really seem chatbot as such? As hard to read as chatbot output, tho.
@davidgerard @ariadne Folks start off speaking like an LLM and thinking like a person, but eventually end up thinking like an LLM but still speaking with the sort of delirious fervour that only a human could conjure
-
@whitequark i agree on super-small models, but all they really need is language -> structured data and back. there are certainly 8B parameter models that can do that well enough.
-
@whitequark i agree on super-small models, but all they really need is language -> structured data and back. there are certainly 8B parameter models that can do that well enough.
@ariadne i acknowledge the legitimate use case you're talking about. my issue is that we're wholly unprepared for the illegitimate ones in a way that probably outweighs the legitimate utility
-
there is, incidentally, no actual evidence to support anything steve yegge is saying here: https://yegge.ai/essays/model-welfare/
@ariadne I enjoyed his rants about 15 yeas ago, but he really seems to have lost the plot
-
@ariadne i acknowledge the legitimate use case you're talking about. my issue is that we're wholly unprepared for the illegitimate ones in a way that probably outweighs the legitimate utility
@whitequark idk, spammers have been using markov chains, seq2seq, etc for years before LLMs gained any interest beyond being a shitpost, the spam problem isn't new, and the disinformation problem really breaks down to what these larger models can do in terms of making convincing fakes, etc
-
there is, incidentally, no actual evidence to support anything steve yegge is saying here: https://yegge.ai/essays/model-welfare/
@ariadne That's bonkers on so many levels. Does he believe his interactions with "The Model" are all routed to the same set of GPUs? And that no requests are handles in-between? Or even that it is the same model version answering him? Does he think distributed random forest clusters predicting warehouse demand become depressed when a pallet is lost? So many questions.
-
if you are going to claim that models have emotion, then you had best have evidence to support that claim.
no serious researcher has yet to make this claim.
@ariadne The burden-of-proof, appeal-to-novelty proposition that LLMs might have emotions statistically approximates to Geoffrey Hinton "getting high on his own supply".
-
@davidgerard @ariadne Folks start off speaking like an LLM and thinking like a person, but eventually end up thinking like an LLM but still speaking with the sort of delirious fervour that only a human could conjure
@jsbarretto @davidgerard @ariadne this is one of the things we don't get — if you want the cognitive equivalent of malware we can hook you up for free, and it doesn't even turn you into a drooling slopmonger, just a coyote. why is he paying 3000 smackeroos a month to huff chatgpt fumes and barf directly into a git repo
-
@whitequark idk, spammers have been using markov chains, seq2seq, etc for years before LLMs gained any interest beyond being a shitpost, the spam problem isn't new, and the disinformation problem really breaks down to what these larger models can do in terms of making convincing fakes, etc
@ariadne @whitequark This is where I get to plug an old friend's website.
https://www.spamradio.com/index/welcome_to_spamradio.html -
@jsbarretto @davidgerard @ariadne this is one of the things we don't get — if you want the cognitive equivalent of malware we can hook you up for free, and it doesn't even turn you into a drooling slopmonger, just a coyote. why is he paying 3000 smackeroos a month to huff chatgpt fumes and barf directly into a git repo
@atax1a @jsbarretto @davidgerard idk, having seen quite a few people go down this path now, i think it's the tight iteration loop leading to frequent dopamine hits that cooks their brain ultimately
-
@atax1a @jsbarretto @davidgerard idk, having seen quite a few people go down this path now, i think it's the tight iteration loop leading to frequent dopamine hits that cooks their brain ultimately
@ariadne @atax1a @jsbarretto he's the most effective developer he's ever been, just ask him
-
@ariadne @atax1a @jsbarretto he's the most effective developer he's ever been, just ask him
@davidgerard @atax1a @jsbarretto mmm, narcissism
-
@atax1a @jsbarretto @davidgerard idk, having seen quite a few people go down this path now, i think it's the tight iteration loop leading to frequent dopamine hits that cooks their brain ultimately
@ariadne @jsbarretto @davidgerard this is a plot point in Strange Days
-
@whitequark i agree on super-small models, but all they really need is language -> structured data and back. there are certainly 8B parameter models that can do that well enough.
@ariadne @whitequark sigh... on-prem models. the false "democratization" bollock. alas, the stolen valour (epistemic injustice) remains... "Sorry doesn't put fingers back on the hand, Marge!" And who pays for the recurring RLHF training cycles, and probably steals more data to stave off the model collapse?
-
@ariadne @whitequark sigh... on-prem models. the false "democratization" bollock. alas, the stolen valour (epistemic injustice) remains... "Sorry doesn't put fingers back on the hand, Marge!" And who pays for the recurring RLHF training cycles, and probably steals more data to stave off the model collapse?
@bms @whitequark not interested in debating reactionary bullshit, sorry. you can train a language model based on wikipedia's database dump in your homelab. these models also already exist.
