I've been saying "if AI is making you so productive then where is all this great new software" and I guess the answer is the software is out there it's just not great, it's terrible, and nobody is using it
-
I asked about the domain, we could be talking about infinite sets, finite sets, where the alphabet is infinite, lossy and lossless and other kind of nuances here. Could have been asking about quantum mechanics here and error correction.
You got specific and attempted to portray a belief I never stated like so as well.
I was more open to things I may not have considered and read up on them. You wanted to try and flex, okay, that's fine.
> I get why you don't _want_ to answer, since the answer proves that with the laws of physics in this universe LLMs don't store copies of their training data.
Which is you speculating about things without evidence here.
To be pedantic here as well:
> Oh this isn't theory
I mean, I was off with measure theory but I guess it belongs to information theory here so not true.
> You're trying to argue against what amounts to natural laws, from your own lack of knowledge about the area.
No, I just outlined the works and I had cited work by academics who have done research in this area and have shown claims that it is able to be reconstructed and leading to the conclusion that they are encoding the data in some form.
@ahto "they are encoding the data in some form."
Yes, they're creating lossy memories from the ingested training. That's what human brains also do.
Neither of us "store copies".
-
"Shannon's theorem is *specifically* about storage"
Yes, that's the fact that disproves LLMs storing their training data verbatim. This is how you make a logic proof.
"let's go back to your original extraordinary claim that neural networks learn like human brains: where's your proof? Your peer-reviewed proof complete with reproducible results? Because they really are nothing alike."
Here's Google's paper from 2017 that is behind all the LLM architectures today:
https://arxiv.org/abs/1706.03762
Here's an explanation to how that's similar to human brains:
I wrote my first back-propagating neural network in 1993. You?
> Yes, that's the fact that disproves LLMs storing their training data verbatim. This is how you make a logic proof.
Really? Here's a type of lossless storage that respects Shannon's theorem while storing verbatim data: I scrape X data off the internet, I can keep only Y data where Y < X. I throw away all non-copyrighted material until I get only Y data. Shannon's theorem was respected and I have verbatim data that I am now distributing without consent, attribution or compensation.
> https://
arxiv.org/abs/1706.03762This contains exactly zero proof that LLMs and brains operate alike.
-
> Yes, that's the fact that disproves LLMs storing their training data verbatim. This is how you make a logic proof.
Really? Here's a type of lossless storage that respects Shannon's theorem while storing verbatim data: I scrape X data off the internet, I can keep only Y data where Y < X. I throw away all non-copyrighted material until I get only Y data. Shannon's theorem was respected and I have verbatim data that I am now distributing without consent, attribution or compensation.
> https://
arxiv.org/abs/1706.03762This contains exactly zero proof that LLMs and brains operate alike.
> https://www.
brown-tth.com/post/the-neural-network-in-our-heads-how-transformer-architectures-mirror-the-human-brainFrom your second link:
> Still, at the end of the day, the brain is far more complex than even the most advanced neural networks today. The brain has 100 billion neurons and 100 trillion connections, compared to Transformers with billions of parameters. The brain also develops its connections over years of growth and learning, whereas Transformers are trained for a limited time on a fixed dataset. The brain is the result of evolution, while humans design Transformers. As researchers put it: “the brain is trained with a recurrent architecture and on a relatively small amount of grounded sentences, while transformers are trained with a massive feedforward architecture and on huge text databases.”
Did you actually read it before posting it?
-
@ahto "they are encoding the data in some form."
Yes, they're creating lossy memories from the ingested training. That's what human brains also do.
Neither of us "store copies".
Oh ffs - This is embarrassing for you dude but thank you for finally coming around to admitting it.
Would replication be a better word instead of copy? Given that exact copies and lossy versions are reconstructable and resembling training material.
However, your argument amounts to the napster one ("But sir, it is an MP3, it is a different from the original"), this not something that is entertained seriously.
Many people are aware of that this happens. Upon review, I did think about the interpretation of "copy" in this conversation and entertained both versions of it being "lossless" and "lossy".
However, both are covered here and I think what this argument culminates to is that:
You are comfortable with the idea of LLMs having some lossy encoding (regardless if it is minor or not or if it can reconstruct the original, you seem to value an excuse for whatever reason).
All this under the guise that "it is like how we learn" which you are lacking some definitive proof here.
I'll chalk up that some of the papers you linked have made some interesting points and have value to how we learn but they are not remotely providing the assertion you are making here.
---
Look,
If you want to use code that you did not write, from a machine made by someone who didn't ask for majority of the authors consent to use, under an excuse that it is 'lossy' - Fine, I don't control you.
However, people probably aren't going to respect you because you clearly do not respect other people or their work.
-
> https://www.
brown-tth.com/post/the-neural-network-in-our-heads-how-transformer-architectures-mirror-the-human-brainFrom your second link:
> Still, at the end of the day, the brain is far more complex than even the most advanced neural networks today. The brain has 100 billion neurons and 100 trillion connections, compared to Transformers with billions of parameters. The brain also develops its connections over years of growth and learning, whereas Transformers are trained for a limited time on a fixed dataset. The brain is the result of evolution, while humans design Transformers. As researchers put it: “the brain is trained with a recurrent architecture and on a relatively small amount of grounded sentences, while transformers are trained with a massive feedforward architecture and on huge text databases.”
Did you actually read it before posting it?
@gabrielesvelto Yes? I'm sorry, maybe you're trying to move the goalposts?
-
Oh ffs - This is embarrassing for you dude but thank you for finally coming around to admitting it.
Would replication be a better word instead of copy? Given that exact copies and lossy versions are reconstructable and resembling training material.
However, your argument amounts to the napster one ("But sir, it is an MP3, it is a different from the original"), this not something that is entertained seriously.
Many people are aware of that this happens. Upon review, I did think about the interpretation of "copy" in this conversation and entertained both versions of it being "lossless" and "lossy".
However, both are covered here and I think what this argument culminates to is that:
You are comfortable with the idea of LLMs having some lossy encoding (regardless if it is minor or not or if it can reconstruct the original, you seem to value an excuse for whatever reason).
All this under the guise that "it is like how we learn" which you are lacking some definitive proof here.
I'll chalk up that some of the papers you linked have made some interesting points and have value to how we learn but they are not remotely providing the assertion you are making here.
---
Look,
If you want to use code that you did not write, from a machine made by someone who didn't ask for majority of the authors consent to use, under an excuse that it is 'lossy' - Fine, I don't control you.
However, people probably aren't going to respect you because you clearly do not respect other people or their work.
"people probably aren't going to respect because you clearly do not respect other people or their work."
The anti-AI fanatics are exactly like antivaxxers. They believe they know the Truth and that all others agree with them except for those in some big conspiracy.
You're just wrong.
(You also don't seem to know that lossy encoding doesn't recreate the source verbatim. This fits with you seemingly having entered this debate without actually having any knowledge about the subject matter. Cue climate denier comparison)
-
"people probably aren't going to respect because you clearly do not respect other people or their work."
The anti-AI fanatics are exactly like antivaxxers. They believe they know the Truth and that all others agree with them except for those in some big conspiracy.
You're just wrong.
(You also don't seem to know that lossy encoding doesn't recreate the source verbatim. This fits with you seemingly having entered this debate without actually having any knowledge about the subject matter. Cue climate denier comparison)
Okay, you can believe whatever you want
.The conclusion is funny but quite a bit sad on your front. I do sincerely hope you reflect on this whole thing but I doubt you will.
-
If you want to make completely different points to the one answered - sure!
Has any artist ever compensated another from having looked at their paintings while learning to draw?
Has any budding coder ever compensated others when having studied their code to learn how to do things?
I'm all for lambasting shitty tech bro AI companies, but that's not the same as claiming that any and all LLM usage is bad. I suggest looking at Mistral AI as a european company that's building datacenters using fully renewable energy and ethically sourced data.
As artists who will happily share what we have with others and economically support other artists when possible, your attempt at an argument is sickening.
Art is not a transaction, you talk to the artists, they teach you how things work, you figure out things and you decompose their work, you create, you have joy, you get frustrated, it has meaning, it has purpose, it is the journey in on itself. By learning how an artist tackles a problem you are not literally ripping off their art and grafting it ransom letter-style into some other art with pieces of other artists.
You say you are a demoscener. What the fuck happened to you? To the joy of creation, to seeing the machine come alive, to coming along with friends and have some dude draw some graphics while another friend bangs the tunes and have everyone coalesce into a beautiful piece of work being orchestrated by a machine the public at large forgot? what happened about telling a story, about doing art where the computer is merely the medium? what happened to that? Who hurt you?
Or it did not happen, because as with many people out there, y'all saw the demoscene not as an art, and just as a way to show off and measure your own e-peens for some cash prize while internalizing crunch culture that game publishers oh-so-desperately craved to normalize in the 90s?
The future is here now, old man, and the future gives no shits about slavery-plagiarism machines y'all try to shove down people's throats. People don't want software for the sake of having software.
By mere definition there is no such thing as an ethical LLM or image generator.
-
As artists who will happily share what we have with others and economically support other artists when possible, your attempt at an argument is sickening.
Art is not a transaction, you talk to the artists, they teach you how things work, you figure out things and you decompose their work, you create, you have joy, you get frustrated, it has meaning, it has purpose, it is the journey in on itself. By learning how an artist tackles a problem you are not literally ripping off their art and grafting it ransom letter-style into some other art with pieces of other artists.
You say you are a demoscener. What the fuck happened to you? To the joy of creation, to seeing the machine come alive, to coming along with friends and have some dude draw some graphics while another friend bangs the tunes and have everyone coalesce into a beautiful piece of work being orchestrated by a machine the public at large forgot? what happened about telling a story, about doing art where the computer is merely the medium? what happened to that? Who hurt you?
Or it did not happen, because as with many people out there, y'all saw the demoscene not as an art, and just as a way to show off and measure your own e-peens for some cash prize while internalizing crunch culture that game publishers oh-so-desperately craved to normalize in the 90s?
The future is here now, old man, and the future gives no shits about slavery-plagiarism machines y'all try to shove down people's throats. People don't want software for the sake of having software.
By mere definition there is no such thing as an ethical LLM or image generator.
"You say you are a demoscener. What the fuck happened to you?"
Still releasing content, just as I've always released content for others to peruse as they see fit.
"y'all try to shove down people's throats"
I am? Strange - I'm positive that I wrote that me and my peers _choose_ to use LLMs, for ourselves, since we see benefits from doing so.
My guess is that you belong to the antivaxxer-like mindset. No facts in the world will ever sway your conviction, because you _know_ the Truth.
-
this post has made all the AI users Big Mad
@eniko thanks for always giving us the best posts to make blocklists about.
And telling us which ones attract the worse kind of people I would identify as "burnable" -
@gabrielesvelto LLMs don't "store copies" when they train. What happens in their neural networks is very similar to what happens in a human brain when learning.
"run of the mill": https://mistral.ai/news/our-contribution-to-a-global-environmental-standard-for-ai/
@troed @gabrielesvelto "very similar to the human brain" is a major missunderstanding of a human brain.
Retract your propaganda, post less, read more. -
@troed @gabrielesvelto "very similar to the human brain" is a major missunderstanding of a human brain.
Retract your propaganda, post less, read more.And in what way is your gut feeling about how the brain works relevant here?
-
And in what way is your gut feeling about how the brain works relevant here?
@troed Are you a fucking idiot? Wait no you already answered that.
You're a developer, not a neurologist. And by the looks of it, you haven't even taken a basic course at a university or read half a book.
It may not be my career now, but I aimed it to be my career. And this is -basics- of how not to treat science as a yes or no.
Brain is still very much a black box of mysteries, for reasons related to actual functionality, not to reasons related to probability.
Get a grip, fool. -
@troed Are you a fucking idiot? Wait no you already answered that.
You're a developer, not a neurologist. And by the looks of it, you haven't even taken a basic course at a university or read half a book.
It may not be my career now, but I aimed it to be my career. And this is -basics- of how not to treat science as a yes or no.
Brain is still very much a black box of mysteries, for reasons related to actual functionality, not to reasons related to probability.
Get a grip, fool.@troed I would explain BY DETAIL a bunch of shit but honestly, I like to read, not to post, as you told someone else.
Apply your bullshit philosophy, go find a book, someone more intelligent than you explaining something, or anything inbetween. I tend to do that, and I become better by doing that.
This is a recommendation. Which I shouldn't be giving to someone so fucking pretentious as you, but there you have it. -
@troed Are you a fucking idiot? Wait no you already answered that.
You're a developer, not a neurologist. And by the looks of it, you haven't even taken a basic course at a university or read half a book.
It may not be my career now, but I aimed it to be my career. And this is -basics- of how not to treat science as a yes or no.
Brain is still very much a black box of mysteries, for reasons related to actual functionality, not to reasons related to probability.
Get a grip, fool.@Nawer_Rapter *hugs*
I suggest Susan Blackmore's "Consciousness: An introduction". Besides that you probably want to read Google's seminal paper "Attention is all you need", coupled with a dumbed down explanation of how that architecture is similar to human brains:
And yes, I am a developer. In 1993 I wrote my first back-propagating neural network.
I'm sorry you didn't get the career you wanted.
-
@Nawer_Rapter *hugs*
I suggest Susan Blackmore's "Consciousness: An introduction". Besides that you probably want to read Google's seminal paper "Attention is all you need", coupled with a dumbed down explanation of how that architecture is similar to human brains:
And yes, I am a developer. In 1993 I wrote my first back-propagating neural network.
I'm sorry you didn't get the career you wanted.
@troed buddy i got the career I wanted in the end, cuz all that shit was full of people attempting to only make money and not making a better world.
I found out the way to make a better world is unrelated to companies and institutions that have been compeltely overtaken by neoliberalism.
I've already taken the classes, that is an -introduction-. Probably to the physiology of the brain (as most are) through the lenses of that it -explains- consciousness.
Fun fact, there's no consensus on that. -
@troed buddy i got the career I wanted in the end, cuz all that shit was full of people attempting to only make money and not making a better world.
I found out the way to make a better world is unrelated to companies and institutions that have been compeltely overtaken by neoliberalism.
I've already taken the classes, that is an -introduction-. Probably to the physiology of the brain (as most are) through the lenses of that it -explains- consciousness.
Fun fact, there's no consensus on that.@Nawer_Rapter So now that you realized that I know more than you on the subject matter, let me reiterate:
Read more, post less.
-
@Nawer_Rapter So now that you realized that I know more than you on the subject matter, let me reiterate:
Read more, post less.
@troed Okay, let's just put the things neatly in this one:
-Consciousness is still heavily discussed, mostly cuz it's ending up on philosophy as hard sciences can't demonstrate anything clearly.
-We can determine some aspects of behavior, not all of them, and we don't know if it's related to randomness or something else entirely.
-Last time I updated on this was 2024, when all the AI bullshit was happening, to be able to point it out.I won't spend my time on sources cuz my god I'm tired
-
@Nawer_Rapter So now that you realized that I know more than you on the subject matter, let me reiterate:
Read more, post less.
... ah yet another anti-AI fanatic that uses the block function to solve their cognitive dissonance. I'll just paste this here then since I had already written a reply:
***
@Nawer_Rapter "I won't spend my time on sources"Ok then.
I fully agree consciousness isn't a solved problem. I'm also of the opinion that it will boil down to dualism vs monism. My own opinion is that there's no difference between a human and a P-Zombie. Today's LLMs are incapable of getting to that point, but an architecture like Hope(1) might.
However, that wasn't the point you decided to jump in and telling me to fuck off about - it's that the Transformer architecture learns similarly to the human brain, which it does.
"Specifically, Transformers are similar to the cerebral cortex, the part of the brain responsible for language, thought, perception, emotion, and planning." (from the previous link).
Anti-AI fanatics tend to not know anything about the subject, yet be extremely vocal about them being Right and everyone else being Wrong. I find that quite amusing. At least you should be able to read a paper on the LLM architectures and see the similarity to what you studied.
-
RE: https://mastodon.social/@jplebreton/116709297088742776
I've been saying "if AI is making you so productive then where is all this great new software" and I guess the answer is the software is out there it's just not great, it's terrible, and nobody is using it
@eniko I think most of the software being written is bespoke, made for an individual, family, or company.
I've been writing software using AI help for about a year now. Only in since around the beginning of the year have I seen it be able to be accurate enough to expand it's usage significantly.
But you asked about software. I have a 100% local AI assistant for my family that does all sorts of things now (I most recently added budgeting using Actual on the backend, HUGE help). I have bug bounty box that is finding vulnerabilities and getting paid for it. I have a autonomous AI project box that started with Moltbook, but now has several dozen agents, including executing x402 transactions (and getting paid for it). I have a automated media server that has a imdb-like GUI where my family can request files to be downloaded.
But my pride an joy is how well I've got my homelab running all the tools I've always wanted. Forgejo with all the same security controls I have in my day job (which actually blocked malware this week!) My family has SSO to all our apps with Authentik. I've got NextCloud and Immich (with local AI image analysis) for files and pictures. I am running a rich monitoring suite with Otel, Uptime Kuma, Beszel, Ntfy, and others. All my projects are documented in Plane. And I have a custom trouble ticking app for my family and AI agents to report issues.
