My first job was building out the first mega-datacenters.
-
@cwebber that's an interesting point of view. I mean, of course the current datacenter craze is complete madness, but it seems you consider an anti-pattern the concept of datacenter itself. Why is it so? What do you suggest as an alternate solution to the problems data centers try to solve?
There's a very simple and actually very affordable alternative to data centers, both commercially and consumer: Self hosting.
Do you have a router in your home? Congratulations, you've got more computing resources than you'll ever need for your own little soapbox on the web as well as sending & receiving email.
For the moment it's too much of a configuration challenge for john-and-jane-doe average, but that's a software problem, that's very much solvable.
-
There's a very simple and actually very affordable alternative to data centers, both commercially and consumer: Self hosting.
Do you have a router in your home? Congratulations, you've got more computing resources than you'll ever need for your own little soapbox on the web as well as sending & receiving email.
For the moment it's too much of a configuration challenge for john-and-jane-doe average, but that's a software problem, that's very much solvable.
Small and mid sized businesses used to host their very own compute infrastructure, some 30 years ago. I was there, I worked summer jobs in their IT departments
IBM System/36, AS/400, Novell Netware, dBase/Clipper… those were the staples of the times, you could find at least one of them, but often several in most mid-sized businesses in Europe and North America.
-
@farfalk Datacenters are concentrations of power. Anytime a datacenter is involved, it's a sign of power centralization. The rise of datacenters corresponds with the death of p2p and other visions of a more decentralized internet.
-
@cwebber @farfalk like, I can appreciate some of the advantages of having them. Like you could get more computer per watt, maybe. I think valuable research is done with supercomputers and modern, more modular approaches to big data. But we could do way more with way fewer datacenters if these weren't used as a way to paywall functionality at the server side. The move to the cloud almost makes me miss when my problem was Cubase requiring a USB dongle.
@thomasjwebb @cwebber @farfalk there is always a "but sometimes" so maybe we can take it as a given that for any strident statement in short form chat there isn't all the nuance about exceptions.
The overall direction seems right to me though. We've got a not insignificant ipv6 deployment for residential use, where is the Cobalt Qube of personal computing? There is no good *technical* reason I shouldn't be able to host my personal email on my own domain, at home, on my own computer, along with a website or whatever. Big monopolistic platforms, which require huge datacenters and complex tech stacks, are an antipattern.
-
@cwebber @farfalk I think it more corresponds to the death of personal computing as it was? People don't have desktops anymore and barely have laptops other than for work? Which is a problem for p2p? Seems like most people's decentralized/federated nodes for things are hosted in data centers? All question marks because just speculating.
-
@thomasjwebb @cwebber @farfalk there is always a "but sometimes" so maybe we can take it as a given that for any strident statement in short form chat there isn't all the nuance about exceptions.
The overall direction seems right to me though. We've got a not insignificant ipv6 deployment for residential use, where is the Cobalt Qube of personal computing? There is no good *technical* reason I shouldn't be able to host my personal email on my own domain, at home, on my own computer, along with a website or whatever. Big monopolistic platforms, which require huge datacenters and complex tech stacks, are an antipattern.
@raven667 @cwebber @farfalk sure I just don’t want to be seen as someone who hasn’t considered the obvious counterpoints. I have the “always include depth-first nuance” kind of autism and ocd. But yeah I think if we design protocols right, maybe people won’t even have to self-host in many cases. It could be some truly p2p stuff than can run on the client.
-
My first job was building out the first mega-datacenters. 2005-2007, I was a datacenter assistant monkey working from Google working somewhere in the Chicago suburbs, swapping out hard drives and ram and writing shell scripts, as myself and my friends unknowingly laid down the prototype for the kinds of datacenters we all see today.
And so it is with some significant expertise that I say:
Fuck datacenters. Datacenters are an anti-pattern.
@cwebber The main motivations for using data centers are security, reliability, flexibility, and efficiency. A data center can manage power consumption much better than your typical on premises installation. Infrastructure can be shared by many users, which allows dynamic scaling without having to over provision on premises. It also allows users to share expert reliability and cybersecurity engineers.
-
@farfalk Datacenters are concentrations of power. Anytime a datacenter is involved, it's a sign of power centralization. The rise of datacenters corresponds with the death of p2p and other visions of a more decentralized internet.
-
My first job was building out the first mega-datacenters. 2005-2007, I was a datacenter assistant monkey working from Google working somewhere in the Chicago suburbs, swapping out hard drives and ram and writing shell scripts, as myself and my friends unknowingly laid down the prototype for the kinds of datacenters we all see today.
And so it is with some significant expertise that I say:
Fuck datacenters. Datacenters are an anti-pattern.
@cwebber
huhinteresting.
about that same time;
had 3 interviews with the goog
as a 'linux beige box wrangler'
at a big campus dc in northern va
my linux clue-kit was deep & shallow at the same time
they passed me over
I was bummed, bigly
then it all changed
-
Small and mid sized businesses used to host their very own compute infrastructure, some 30 years ago. I was there, I worked summer jobs in their IT departments
IBM System/36, AS/400, Novell Netware, dBase/Clipper… those were the staples of the times, you could find at least one of them, but often several in most mid-sized businesses in Europe and North America.
@datenwolf @farfalk @cwebber lots of financial institutions still have private datacenters, and still run IBM mainframes. Just easier to manage from a security perspective.
-
@cwebber @farfalk How would you approach defining the threshold of concentration past which it's undesirable? The only obvious approach I can think of is "it's too high if there's a positive feedback loop", but that's both not really knowable and probably too low, given that we ~started from not much concentration and arrived at current, clearly undesirably high, levels thereof.
-
@farfalk @cwebber Really look at “the problems data centers try to solve”. At face value, LLMs and other “AI” are not functional or even profitable by themselves, but they are the supposed reason for the data center boom. But there’s strong evidence that the boom is driven by market manipulation for the hardware, not organic demand for its work. Further, the face value function of “AI” is to extract short term cash value while denying resources to humans. That is the secondary problem the centers try to solve (first being fraudulent investment in the centers themselves). That’s why framing it as “what’s your alternative” is a mistake.
@Moss @farfalk @cwebber sorry I can't quite understand your point. I am sure you use the internet for distributed software; data centers per se allow networking. Having products controlled by monopolistic rent seeking companies that haven't a better business plan than ads is a political problem, not so different from rail roads/oil distribution; political solutions will work when we focus on democratic power not individual purity. Bubbles happen when $ is unregulated by work/life/democracy.
-
My first job was building out the first mega-datacenters. 2005-2007, I was a datacenter assistant monkey working from Google working somewhere in the Chicago suburbs, swapping out hard drives and ram and writing shell scripts, as myself and my friends unknowingly laid down the prototype for the kinds of datacenters we all see today.
And so it is with some significant expertise that I say:
Fuck datacenters. Datacenters are an anti-pattern.
@cwebber I did this in Oregon. The town looks nothing like its old self.
-
@Moss @farfalk @cwebber sorry I can't quite understand your point. I am sure you use the internet for distributed software; data centers per se allow networking. Having products controlled by monopolistic rent seeking companies that haven't a better business plan than ads is a political problem, not so different from rail roads/oil distribution; political solutions will work when we focus on democratic power not individual purity. Bubbles happen when $ is unregulated by work/life/democracy.
@jayalane @farfalk @cwebber “data centers = networking” is the same as saying “data center water and power consumption = indoor plumbing and wiring.” We had global networking long before the present eruption of resource-hogging new data centers. My point was exactly as stated: the so-called problems that the DCs are supposed to solve are fake. A narrative beard for fraud and global resource extraction.
-
@jayalane @farfalk @cwebber “data centers = networking” is the same as saying “data center water and power consumption = indoor plumbing and wiring.” We had global networking long before the present eruption of resource-hogging new data centers. My point was exactly as stated: the so-called problems that the DCs are supposed to solve are fake. A narrative beard for fraud and global resource extraction.
@Moss @farfalk @cwebber I certainly don't think we need new data centers but my mastodon stuff and email and backups are all on VMs in data centers. The LLM devs could do the GenAI/LLM on a nice small experimental development scale of the new capabilities without building so much because they have been given trillions and have to spend it. The greed and need to spend distort the whole process. Like all the dark fiber around 2000 or Cisco stock price. 1/2
-
@Moss @farfalk @cwebber I certainly don't think we need new data centers but my mastodon stuff and email and backups are all on VMs in data centers. The LLM devs could do the GenAI/LLM on a nice small experimental development scale of the new capabilities without building so much because they have been given trillions and have to spend it. The greed and need to spend distort the whole process. Like all the dark fiber around 2000 or Cisco stock price. 1/2
@Moss @farfalk @cwebber As far as I know we never had global networking without small data centers post 1990, and even then you had phone company premises. You need interconnect and buildings are cheaper than say satellites. And the data centers my VMs run in are probably more resource efficient than when they were in my basement. The problem isn't "data centers" but "investors have the unopposed power to waste trillions and ruin things for the multitudes due to too much unregulated money." /2
-
@Moss @farfalk @cwebber As far as I know we never had global networking without small data centers post 1990, and even then you had phone company premises. You need interconnect and buildings are cheaper than say satellites. And the data centers my VMs run in are probably more resource efficient than when they were in my basement. The problem isn't "data centers" but "investors have the unopposed power to waste trillions and ruin things for the multitudes due to too much unregulated money." /2
-
Working in that environment, seeing as Google rolled out the idea of "cloud computing" meaning "you have no involvement or agency in your computing because we do it for you" radicalized me for much of the work of my career.
It was one thing to run a datacenter to index the world's public web information. I understood that, it made sense.
But watching as Google and Apple co-developed the idea that computers, which I cared about, got abstracted into toys and jewelry that had all your key computing done in a way you had no agency over... where I saw firsthand the kinds of churn of resources necessary to keep these things going, it made me want to fight for a different computing future.
@cwebber yes, this!
I said that when I heard of that cloud thing for the first time. They laughed at me. People still tell me that it’s the new and now only way of doing things and required for work 🤮
-
@cwebber @thomasjwebb I remember "datacenter" starting as "colocation hoster" - you rentet rackspace or several racks with redundant power supply, internet link, packed in some pizzaboxes and a router, and there you go. Physical safety was better than the rack with dev servers in the basement, so what else could you ask for?
@Reinald @thomasjwebb @cwebber colocation of your own stuff is ok
-
@vex @cwebber @farfalk To take a somewhat extreme threshold, centralizing more compute power than a person or a few can provide (i.e. populatization of mechanical calculators) was a likely irreversible transition. Same can be said about increases in computing power of various nonnetworked calculators and eventually computers. I think same can be said about networking -- initially precisely so that you can share time of a computer that's shared across many users.
Timesharing is a subjectively interesting case: it's a situation where the reason to centralize --load smoothing-- is actually reasonable (i.e. is not due to coordination failures). But I can't really point out why this is more of centralization than every earlier increase in computing power of computers, which caused more computing power to be centralized in individual locations, and at least initially pretty few of those.
