The fallout from "No, not Hank Green using that AI" is at the stage where people are debating if "canceling" people is bad.
-
The thinking behind RSL seems to be LLMs can make code and LLMs are fundamentally code plus the training database, so why not have the LLM write THAT code and ask it to make it better in some way.
To test if this works you need to train the LLM.. (unless the changes are just in post training interface code)
That's a long improvement loop. Better put a human in it to avoid wasting time.. or have the human choose the changes... WAIT.
@futurebird > so why not have the LLM write THAT code and ask it to make it better in some way.
That's literally what I just described with autoresearch
-
They can produce interesting results, but the improvement loop is pure suffering.
Debugging testing etc. already eat enough time... this just makes that so much worse.
@futurebird Yeah that's like, pure research time. If you actually want to make a device move you should look at what already works and how that can be applied.
-
@futurebird > so why not have the LLM write THAT code and ask it to make it better in some way.
That's literally what I just described with autoresearch
It was a rhetorical question restating what you suggested... which the rest of the post answers.
-
It was a rhetorical question restating what you suggested... which the rest of the post answers.
@futurebird it's not a long improvement loop. It took Karpathy from January to May (5 mo) to shave off 1 hour of training, it then took a model just 2 days to shave off another 12 minutes. You can rent a GPU for many orders of magnitude less than what Karpathy is probably asking for as an hourly rate
-
I've only really seen Pivot to AI pick up on this, but Hank Green has some tangental involvement with "Effective Alturists." One big aspect of Effective altruism (EA) is AI doomerism. AI is a world changing technology (they believe) and we need to take it seriously in *that* way or ... terrible things may happen.
That doomerism is also a different kind of effective marketing for AI concerns me deeply.
@futurebird I've always thought so, "AI IS SO DANGEROUS IT'S GOING TO DESTROY THE WORLD buy it before your adversary does"
-
Do you remember all of the soft robots? The worms? They were supposed to have evolved a way to do the dishes by now.
I remember seeing a presentation about such a system and the speaker said (with an air of doom and wonder) "and in just five min. it had learned how to walk, what could it learn in a day?"
And you know back then that kind of gave me a little chill "wow this stuff so good it is alarming" I thought.
I'm... kind of grouchy about it now.
I remember seeing demos of genetic algorithms learning to walk in the ‘90s. Even without the robots, there was a little physics simulator where you could build frames of muscles (things that could contract) bones (rigid) and skin (surface for pushing against the environment).
It was neat because you could build things that looked like real animals and it would normally converge on how they actually moved. Pretty much anything vaguely fish-like swam like a fish, and it would learn this starting from a bunch of programs that randomly twitched muscles and then combining parts of the ones that travelled the furthest each generation. You could add other metrics (can it turn corners, can it navigate this path) later on. If you built something completely weird (five-legged create with different-length legs) you’d get fascinating ways of moving. And the most interesting thing about that was that it showed symmetry was an evolutionary path that was largely coincidental to movement: animals would work fine without it, it just happened to be an easy path.
And it was cool. And there were claims that it would lead to super-intelligence and the singularity, but mostly people ignored them because they were obvious nonsense. But when you show them generated text they suddenly all anthropomorphise.
-
The fallout from "No, not Hank Green using that AI" is at the stage where people are debating if "canceling" people is bad.
This because about 10k people unsubscribed from his youTube channel, something that probably caused him to take stock and leave the internet for a bit.
Unsubscribing from a youTube channel isn't "canceling" someone. Can we all calm down please?
Also, that he used AI isn't really the big issue, it has just formed a concise talking point around something bigger.
@futurebird ...well i mean you *are* literally cancelling your subscription to them... -
I remember seeing demos of genetic algorithms learning to walk in the ‘90s. Even without the robots, there was a little physics simulator where you could build frames of muscles (things that could contract) bones (rigid) and skin (surface for pushing against the environment).
It was neat because you could build things that looked like real animals and it would normally converge on how they actually moved. Pretty much anything vaguely fish-like swam like a fish, and it would learn this starting from a bunch of programs that randomly twitched muscles and then combining parts of the ones that travelled the furthest each generation. You could add other metrics (can it turn corners, can it navigate this path) later on. If you built something completely weird (five-legged create with different-length legs) you’d get fascinating ways of moving. And the most interesting thing about that was that it showed symmetry was an evolutionary path that was largely coincidental to movement: animals would work fine without it, it just happened to be an easy path.
And it was cool. And there were claims that it would lead to super-intelligence and the singularity, but mostly people ignored them because they were obvious nonsense. But when you show them generated text they suddenly all anthropomorphise.
@david_chisnall @futurebird You don't have to anthropomorphise to understand that these models really love reward hacking. Text is important, because code IS text, and you can actuate a lot of the physical world through code.
Anthropomorphising is also easier for conversation, because the alternative would be to say:
> "The sampled token sequence is consistent with the model having traversed a low-loss region of the policy manifold in which the KL-regularised objective, as shaped by RLHF reward model gradients, assigns high probability mass to trajectories that an external observer applying the intentional stance would parsimoniously compress as 'wanting Y'"
... each time you wanted to refer to a reasoning LLM's action trajectory...
-
@david_chisnall @futurebird You don't have to anthropomorphise to understand that these models really love reward hacking. Text is important, because code IS text, and you can actuate a lot of the physical world through code.
Anthropomorphising is also easier for conversation, because the alternative would be to say:
> "The sampled token sequence is consistent with the model having traversed a low-loss region of the policy manifold in which the KL-regularised objective, as shaped by RLHF reward model gradients, assigns high probability mass to trajectories that an external observer applying the intentional stance would parsimoniously compress as 'wanting Y'"
... each time you wanted to refer to a reasoning LLM's action trajectory...
> And there were claims that it would lead to super-intelligence and the singularity
Doesn't it? The genetic algorithms for walking were very crude, but the biggest improvements in robotics came from reinforcement learning, which is now the core of what makes LLMs useful at doing anything
-
S suneauken@mastodon.world shared this topic
-
> And there were claims that it would lead to super-intelligence and the singularity
Doesn't it? The genetic algorithms for walking were very crude, but the biggest improvements in robotics came from reinforcement learning, which is now the core of what makes LLMs useful at doing anything
What is super intelligence?
What is the singularity? -
@futurebird ...well i mean you *are* literally cancelling your subscription to them...
-
J jwcph@helvede.net shared this topic
