no notes
-
@orange_lux @saxnot @dibi58 @karl llms are useful for NLP. Inference is actually relatively cheap, it's the training that's really expensive and resource intensive. We've probably already maxed out LLM capabilities, so most of this training is not useful. Companies keep training because they need to convince investors that infinite growth is possible. What actual gains are being made are coming from architectural changes, not from training.
Basically, "AI data centers" should not exist. Local models can do everything that's needed. If we need to train new models, those need to be balanced against climate goals (basically, don't fucking do it). And LLMs should be removed from basically everything they've been shoved into recently.
If you don't know why LLMs are useful, you shouldn't have to interact with LLMs. Even some of the places where they are useful, they can be used to construct cheaper models.
There are a few things, like correlation across huge data sets, that they're useful for. But even then, simple encoding can give you semantic search, where inference is not necessary or only provides minimal additional benefit.
Yeah, basically, 95-99% reduction in cars and AI. It's basically the same thing.
@Hex @orange_lux @saxnot @dibi58 @karl I suspect you're underestimating the compute being dedicated to running (not training) the models.
I was recently observing a conversation between small startup founders who are buying €100k EGX Station AI workstations to move their developers from Cursor/Claude to local models. The same person seemed both to be questioning the value of all this LLM use by his dev team, at the same time considering buying €100k in hardware to reduce his LLM costs.
-
no notes
@karl Yikes. People will die of heat this week in France
-
@karl Imagine a group of hyper-intelligent beings building an incredibly powerful computer AGI in order to find out the answer to the ultimate question of life, the universe and everything
-
@karl The article has been amended to remove the quote "Editor’s note: This is an updated version of the article. The previous version wrongly quoted Jeff Bezos. The error is regretted."
-
no notes
@karl That's true, but the temperature info graphic is horrendous. It is in absolute temperature and goes from -30°C - 50°C... that's bonkers.
Don't use stupid graphics, if the good ones are out there. (relative Temperature for a day is much better. )
-
no notes
@karl I would like the @EUCommission to take note and do something USEFUL for a while instead of saying just "yes my lord" to whatever tech lobby comes to them.
-
@Hex @orange_lux @saxnot @dibi58 @karl I suspect you're underestimating the compute being dedicated to running (not training) the models.
I was recently observing a conversation between small startup founders who are buying €100k EGX Station AI workstations to move their developers from Cursor/Claude to local models. The same person seemed both to be questioning the value of all this LLM use by his dev team, at the same time considering buying €100k in hardware to reduce his LLM costs.
@sherbang @Hex @orange_lux @saxnot @dibi58 I work at a cloud provider and can confirm that running ai isn't as cheap as it sounds.
My take is that most models are so general purpose that they're very inefficient (versatile, yes, but inefficient). Think "sorting an array through a bogo sort" inefficient. LLMs trained for a specific purpose may be more cost-effective to run long term, but that's not the norm.
-
no notes
@karl no difficult choice
-
@orange_lux @dibi58 @karl technically we don't "need" anything
are llm a useful tech with pros ans cons?
aee gasoline engines a useful tech with pros ans cons?
are paperweighr a useful tech with pros ans cons?but there is no "AI"
no A(G)I was ever invented
the media around llm is detached from reality, frentic, toxic and more positive than the reality warrants -
@orange_lux @saxnot @dibi58 @karl llms are useful for NLP. Inference is actually relatively cheap, it's the training that's really expensive and resource intensive. We've probably already maxed out LLM capabilities, so most of this training is not useful. Companies keep training because they need to convince investors that infinite growth is possible. What actual gains are being made are coming from architectural changes, not from training.
Basically, "AI data centers" should not exist. Local models can do everything that's needed. If we need to train new models, those need to be balanced against climate goals (basically, don't fucking do it). And LLMs should be removed from basically everything they've been shoved into recently.
If you don't know why LLMs are useful, you shouldn't have to interact with LLMs. Even some of the places where they are useful, they can be used to construct cheaper models.
There are a few things, like correlation across huge data sets, that they're useful for. But even then, simple encoding can give you semantic search, where inference is not necessary or only provides minimal additional benefit.
Yeah, basically, 95-99% reduction in cars and AI. It's basically the same thing.
@Hex @orange_lux @dibi58 @karl
> Companies keep training because they need to convince investors that infinite growth is possible.
jup
once again it's capitalism -
@Hex @orange_lux @dibi58 @karl
> Companies keep training because they need to convince investors that infinite growth is possible.
jup
once again it's capitalism@Hex @orange_lux @dibi58 @karl > like correlation across huge data sets, that they're useful for
uh do they?
isn't this a domain already domineered by other ML?dont misunderstand me:
Transformers can see structures noone else can see
but why an LLM for that... feels misplaced -
@orange_lux @dibi58 @karl yeah so?
we had recommendations long before our current gen (pre llm) stuffso?
data science is a trillion dollar industry
yeah there was more than one generation of recommendation algorithm
llm are a new tool and at some things they are better -
@orange_lux @dibi58 @karl yeah so?
we had recommendations long before our current gen (pre llm) stuffso?
data science is a trillion dollar industry
yeah there was more than one generation of recommendation algorithm
llm are a new tool and at some things they are better@orange_lux @dibi58 @karl those amazon recommendation matrices are not cheap either
not really a comparison that is fair but LLM is just a minor evolution in a field with centuries of real life usage, development etc
-
@orange_lux @dibi58 @karl those amazon recommendation matrices are not cheap either
not really a comparison that is fair but LLM is just a minor evolution in a field with centuries of real life usage, development etc
@orange_lux @dibi58 @karl today I don't have the capacity for whataboutism
you're missing the engineering point
there is already enough misinformation out there
-
@orange_lux @dibi58 @karl today I don't have the capacity for whataboutism
you're missing the engineering point
there is already enough misinformation out there
-
@sherbang @Hex @orange_lux @saxnot @dibi58 I work at a cloud provider and can confirm that running ai isn't as cheap as it sounds.
My take is that most models are so general purpose that they're very inefficient (versatile, yes, but inefficient). Think "sorting an array through a bogo sort" inefficient. LLMs trained for a specific purpose may be more cost-effective to run long term, but that's not the norm.
@karl @sherbang @orange_lux @saxnot @dibi58 I know how much it takes to run at least basic models because I'm running local models for experiments. I won't use hosted models because I'm not giving them money or training data. But yeah, capitalists are trying to sell something that doesn't exist and that they don't understand.
I would be entirely unsurprised to find out that even inference on corporate models can't cover costs. To sell "AI" it has to be a thing that just works for everything all the time. It must take no thought. That's incredibly wasteful.
It needs to stop being subsidized, just like cars.
-
no notes
@karl En Politico zal ons even vertellen wat wij moeten doen of laten? And Politico is the instance telling us what to do or not?
-
no notes
Reference, for whoever might care…
https://www.politico.eu/article/europe-choose-ai-climate-goals-data-center-chief-warns/
-
@karl Difícil elección. O morir de calor en pos del beneficio de los ricos o no hacerlo. Dadme por favor 9ms para pensármelo...
@cucufaiter @karl 9 minutes? Yep same with me, spent all my tokens, query still running.
-
@cucufaiter @karl 9 minutes? Yep same with me, spent all my tokens, query still running.
@a_goodall_spaceship @karl miliseconds xD
