just learned about this project erjic that does seccomp+bwrap sandboxing https://codeberg.org/prisixia/erjic the interface is really nicely designed and i believe will fit the needs of my build system perfectly
-
i was hired by LLNL right as spack was transitioning from a messy non-backtracking serial graph solver (our own, in python, grown organically over the decade since spack was created) to clingo (this was a huge risk). the reason we could do this was because package recipes were already declarative, and already described all versions of a package across all architectures at once.
this is super important: we already had a logical model—we just needed a more powerful solver. the solver did not dictate our logic. we applied the solver to our use case.
this is the advantage of protocols. spack was made for LLNL to provide a service to our physicists. these physicists encompass mathematicians and programmers—LLNL does simulations, so like the spack team, they're necessarily applied scientists. the tools development group (TDG) does spack, but our larger org within LLNL computing also developed parallelism libraries, which themselves made use of memory management primitives
-
this is the advantage of protocols. spack was made for LLNL to provide a service to our physicists. these physicists encompass mathematicians and programmers—LLNL does simulations, so like the spack team, they're necessarily applied scientists. the tools development group (TDG) does spack, but our larger org within LLNL computing also developed parallelism libraries, which themselves made use of memory management primitives
the purpose of all of this is to bring deep hardware-dependent concerns all the way up to the application layer, because HPC is generally one of the fields that requires this at all times. but notice something very important—the application layer is not the spack team, nor the RAJA team, but the scientists we serve.
our goal is to make their scientific work more efficient than any alternative. that's how we broke the nuclear fusion net-positive efficiency barrier after 60 years. that's what differentiates LLNL from LANL, who still pretends quantum computing is totally real.
-
the purpose of all of this is to bring deep hardware-dependent concerns all the way up to the application layer, because HPC is generally one of the fields that requires this at all times. but notice something very important—the application layer is not the spack team, nor the RAJA team, but the scientists we serve.
our goal is to make their scientific work more efficient than any alternative. that's how we broke the nuclear fusion net-positive efficiency barrier after 60 years. that's what differentiates LLNL from LANL, who still pretends quantum computing is totally real.
the point of this hagiography is not to claim the US is good at science, or that the spack team is smarter than anyone else (except LANL, who is overrated). in fact, i believe the netherlands is far better at funding science, and i fervently hope their physicists develop better silicon fabrication techniques soon.
the US is in fact terrible at funding science. but government employees are different from military generals or failson politicians. and in this hostile environment with funding swings every 4 years, this is what we arrived at to defend science from being beholden to IBM
-
the point of this hagiography is not to claim the US is good at science, or that the spack team is smarter than anyone else (except LANL, who is overrated). in fact, i believe the netherlands is far better at funding science, and i fervently hope their physicists develop better silicon fabrication techniques soon.
the US is in fact terrible at funding science. but government employees are different from military generals or failson politicians. and in this hostile environment with funding swings every 4 years, this is what we arrived at to defend science from being beholden to IBM
blue gene? that beat kasparov? that machine is still running. it was also the last IBM machine purchased by LLNL, because IBM cannot be relied upon to provide features for any sum of taxpayer money. they are completely incapable of it. the first exascale machine (possibly second, idgaf) is instead a completely modular supercomputer from consumer AMD chips. and spack enables the government to pay AMD a bazillion dollars for bragging rights without getting stuck into a software–hardware monopoly like IBM loves
-
blue gene? that beat kasparov? that machine is still running. it was also the last IBM machine purchased by LLNL, because IBM cannot be relied upon to provide features for any sum of taxpayer money. they are completely incapable of it. the first exascale machine (possibly second, idgaf) is instead a completely modular supercomputer from consumer AMD chips. and spack enables the government to pay AMD a bazillion dollars for bragging rights without getting stuck into a software–hardware monopoly like IBM loves
open source, in the informal sense, is something todd gamblin (spack creator, one of my favorite people in the whole world) convinced the DOE to invest in (like i did at twitter), and he was immediately proven right, because a ridiculous number of other labs are now using spack.
this also means the power relations spack enables (there are always power relations in any org) are now available elsewhere. spack in this view is an institutional and political tool—just like every other package manager.
-
open source, in the informal sense, is something todd gamblin (spack creator, one of my favorite people in the whole world) convinced the DOE to invest in (like i did at twitter), and he was immediately proven right, because a ridiculous number of other labs are now using spack.
this also means the power relations spack enables (there are always power relations in any org) are now available elsewhere. spack in this view is an institutional and political tool—just like every other package manager.
you can and should argue that "b2b vs b2c" codifies deeply corporate-centric profit relations, especially those advanced by VCs and tech startups. in that way, it's a harmful analogy, and it exposes my preconceptual bias from employment (twitter in US tech, and LLNL in US government/academia). i would love to hear ways the "customer" terminology inculcates harmful assumptions we should steer away from. please dunk on me. write screeds of my misdeeds.
-
you can and should argue that "b2b vs b2c" codifies deeply corporate-centric profit relations, especially those advanced by VCs and tech startups. in that way, it's a harmful analogy, and it exposes my preconceptual bias from employment (twitter in US tech, and LLNL in US government/academia). i would love to hear ways the "customer" terminology inculcates harmful assumptions we should steer away from. please dunk on me. write screeds of my misdeeds.
but i like the word "customer" because it's missing from "open source". when maintainers (consider xz-utils) are overwhelmed, they're serving microsoft customers. but those customers can't reach them, and certainly can't pay them. microsoft, and google, and facebook, and certainly amazon, are serving as middlemen and doing a shit fucking job at it
-
but i like the word "customer" because it's missing from "open source". when maintainers (consider xz-utils) are overwhelmed, they're serving microsoft customers. but those customers can't reach them, and certainly can't pay them. microsoft, and google, and facebook, and certainly amazon, are serving as middlemen and doing a shit fucking job at it
i especially seek to destroy google for vengeance, but the entire cloud apparatus runs on not just gnu/linux, but every fucking maintainer in between. more and more it's musl/linux because glibc is monopolistic, yet somehow linus torvalds is the only one getting rich off this shit
-
i especially seek to destroy google for vengeance, but the entire cloud apparatus runs on not just gnu/linux, but every fucking maintainer in between. more and more it's musl/linux because glibc is monopolistic, yet somehow linus torvalds is the only one getting rich off this shit
the end user, living the dream of linux on the desktop, largely uses gnome or KDE. fun fact: arch's
pacmannow has a new type of dependency, that's not listed as optional but can be uninstalled freely. KDE's "baloo" which indexes your entire hard drive and can't be turned off must be uninstalled with-Rddafter nearly every-Syuupgrade. it's not registered insystemctl—it just pops up inps. if you ever provide your location to the weather in the KDE clock, it cannot be removed, not without undocumented changes in~/.config.who is the customer for this? who benefits? who is all this for?
-
the end user, living the dream of linux on the desktop, largely uses gnome or KDE. fun fact: arch's
pacmannow has a new type of dependency, that's not listed as optional but can be uninstalled freely. KDE's "baloo" which indexes your entire hard drive and can't be turned off must be uninstalled with-Rddafter nearly every-Syuupgrade. it's not registered insystemctl—it just pops up inps. if you ever provide your location to the weather in the KDE clock, it cannot be removed, not without undocumented changes in~/.config.who is the customer for this? who benefits? who is all this for?
i was shocked to finally understand the zen of slackware thanks to @miss_rodent's patient explanation. no dependencies? what's the point then? why not build everything by hand? but i slowly began to pull these apart:
- a distro package recipe bundles the code for your distro
- a package manager builds or downloads the package by name (checking signatures and/or checksums)
afaiu, slackware does that just fine. and it is better than building from source by hand, because someone else did the work to build it reliably and install it in the right places. what it doesn't do is enforce a correctness constraint that forces you to pull in packages you didn't explicitly consent to (!!!)
-
i was shocked to finally understand the zen of slackware thanks to @miss_rodent's patient explanation. no dependencies? what's the point then? why not build everything by hand? but i slowly began to pull these apart:
- a distro package recipe bundles the code for your distro
- a package manager builds or downloads the package by name (checking signatures and/or checksums)
afaiu, slackware does that just fine. and it is better than building from source by hand, because someone else did the work to build it reliably and install it in the right places. what it doesn't do is enforce a correctness constraint that forces you to pull in packages you didn't explicitly consent to (!!!)
@hipsterelectron @miss_rodent so what you're saying is the package manager slacks off
-
i was shocked to finally understand the zen of slackware thanks to @miss_rodent's patient explanation. no dependencies? what's the point then? why not build everything by hand? but i slowly began to pull these apart:
- a distro package recipe bundles the code for your distro
- a package manager builds or downloads the package by name (checking signatures and/or checksums)
afaiu, slackware does that just fine. and it is better than building from source by hand, because someone else did the work to build it reliably and install it in the right places. what it doesn't do is enforce a correctness constraint that forces you to pull in packages you didn't explicitly consent to (!!!)
transitive dependencies absolutely raise questions about user consent. and failing to separate the human groups communicating across the dependency graph leads to consent violations by fiat. @ariezra's Industry Unbound used the term "engineering fiat" for intentional corporate violations of their customer's consent and it put words to what i had observed but could never verbalize from google and the rest
-
i was shocked to finally understand the zen of slackware thanks to @miss_rodent's patient explanation. no dependencies? what's the point then? why not build everything by hand? but i slowly began to pull these apart:
- a distro package recipe bundles the code for your distro
- a package manager builds or downloads the package by name (checking signatures and/or checksums)
afaiu, slackware does that just fine. and it is better than building from source by hand, because someone else did the work to build it reliably and install it in the right places. what it doesn't do is enforce a correctness constraint that forces you to pull in packages you didn't explicitly consent to (!!!)
@hipsterelectron @miss_rodent wait, what? Isn’t Slackware rpm-based?
-
@hipsterelectron @miss_rodent wait, what? Isn’t Slackware rpm-based?
@hipsterelectron @miss_rodent I guess not. My use of Slackware was in the very early aughts without reliable access to binary package repositories, so after initial install, I built a lot of things from source.
-
transitive dependencies absolutely raise questions about user consent. and failing to separate the human groups communicating across the dependency graph leads to consent violations by fiat. @ariezra's Industry Unbound used the term "engineering fiat" for intentional corporate violations of their customer's consent and it put words to what i had observed but could never verbalize from google and the rest
and the "correctness" construction of a global shared versioning, which derives ancestrally from shared libraries and the libc, has been bowdlerized and twisted beyond recognition, while the end user has absolutely no mechanism for configuration:
- what if i wanted kde connect without network access? too bad, now the service can ping whoever it wants.
- what if i wanted systemd without the user age record functionality required for OS vendors in pending US federal law? too bad, your distro is US surveillance compliant now.
- what if i wanted linux without "AI"? too bad, we publicized vulnerabilities we knew about years ago and we hate LTS kernels so you're on the backdoored 7.0
-
transitive dependencies absolutely raise questions about user consent. and failing to separate the human groups communicating across the dependency graph leads to consent violations by fiat. @ariezra's Industry Unbound used the term "engineering fiat" for intentional corporate violations of their customer's consent and it put words to what i had observed but could never verbalize from google and the rest
@hipsterelectron @ariezra "Transitive dependencies violate consent" sounds like some purist view about not wanting tainted code present on your machine at all, until you realize what it actually means in practice.
The best examples I know of are polkit and dbus service activation.
There are packages that pull in polkit as a dependency, and when they do, that's not just code present for the program to run. It's literally *installing a backdoor to root* which will be automatically executed to give requesting processes elevated permissions if conditions met in obtuse configuration files (also installed without consent).
Same conceptual thing happens with dbus service activation in general. Mere presence of these service files causes existing dbus to give access to launch things without your consent.
-
and the "correctness" construction of a global shared versioning, which derives ancestrally from shared libraries and the libc, has been bowdlerized and twisted beyond recognition, while the end user has absolutely no mechanism for configuration:
- what if i wanted kde connect without network access? too bad, now the service can ping whoever it wants.
- what if i wanted systemd without the user age record functionality required for OS vendors in pending US federal law? too bad, your distro is US surveillance compliant now.
- what if i wanted linux without "AI"? too bad, we publicized vulnerabilities we knew about years ago and we hate LTS kernels so you're on the backdoored 7.0
because spack was conceived in an environment where the end user is a programmer and not a packager, and used in airgaps (the original sandbox), it had to support injecting deps from the host (compiler, libc) and configuring build-time options, without forcing physicists to also keep up with configure flag renames and build system changes.
a spack spec says "i don't want qt. computer, give me physics without qt." and the computer does what computers are fucking good at and applies the rulesets from spack package recipe maintainers.
-
because spack was conceived in an environment where the end user is a programmer and not a packager, and used in airgaps (the original sandbox), it had to support injecting deps from the host (compiler, libc) and configuring build-time options, without forcing physicists to also keep up with configure flag renames and build system changes.
a spack spec says "i don't want qt. computer, give me physics without qt." and the computer does what computers are fucking good at and applies the rulesets from spack package recipe maintainers.
@hipsterelectron amen to a system that can track wtf you need well enough for airgaps. modern bullshit environments, tell someone you're gonna dev on an airgap'd box and they won't even believe it can be done. and it's not easy.
-
because spack was conceived in an environment where the end user is a programmer and not a packager, and used in airgaps (the original sandbox), it had to support injecting deps from the host (compiler, libc) and configuring build-time options, without forcing physicists to also keep up with configure flag renames and build system changes.
a spack spec says "i don't want qt. computer, give me physics without qt." and the computer does what computers are fucking good at and applies the rulesets from spack package recipe maintainers.
it is a fucking FAILURE if your system requires end users to concern themselves with how packages are built. it is a fucking FAILURE if your system requires end users to recreate your system by hand to avoid surveillance and backdoors. it is a fucking FAILURE if your system cannot handle fucking updating a dependency without invalidating an entire subgraph, let alone the whole fucking graph.
-
it is a fucking FAILURE if your system requires end users to concern themselves with how packages are built. it is a fucking FAILURE if your system requires end users to recreate your system by hand to avoid surveillance and backdoors. it is a fucking FAILURE if your system cannot handle fucking updating a dependency without invalidating an entire subgraph, let alone the whole fucking graph.
it is a fucking FAILURE if your system has such a limited logical representation of package dependencies that you cannot support injecting a user-provided compiler or library without making them build it from scratch within your system. that does not make people safer. it means users will take shortcuts and rely upon someone else's recipe because you have made it impossible for them to use their own trusted tooling
