The coreutils Rust rewrite story is pretty funny.
-
@grumpybozo @lcamtuf afaik the BSD core utils aren't entirely compatible with the gnu core utils, still?
But yeah, there are more permissively licensed versions of the *nix coreutils already; rust uutils is aiming to be a drop-in replacement for the gnu coreutils specifically, though, which means all the gnu-specific extensions and peculiarities. Which, previously, were basically only under the gpl (and some scripts and such can break if you don't have those, so, it's a meaningful difference.)@miss_rodent @lcamtuf Right, there are some variances in command line options, usually in areas not covered by POSIX.
-
-
@lcamtuf very much a Chesterton's Fence kind of situation
@darkuncle tysm for pointing me to this amazing parable, amigos.
️
-
It’s frustrating that POSIX took decades to get APIs that weren’t intrinsically racy, but then higher-level languages that post dated the improved ones implemented equivalents of the old racy APIs. C++ was annoying, they waited until pretty much every platform that supported C++ and had a filesystem implemented the newer APIs and then standardised the filesystem TS with racy ones. I believe Rust is similar, but at least it has cap-std which implements the non-racy versions as an alternative standard library.
@david_chisnall @lcamtuf Well people have opinions: https://mastodon.social/@pid_eins/116459585811044061
-
The coreutils Rust rewrite story is pretty funny.
Coreutils are tools like rm, mv, mkdir, etc. Unlike binutils, this isn't a fertile ground for memory safety bugs. But, the rewrite was completed, and in the spirit of progress, Canonical decided to switch.
But do you know what coreutils are a fertile ground for? Race conditions around file creation, deletion, permission setting, and so on. The original code accounted for decades of hard-learned lessons in that space. The Rust rewrite did not:
https://seclists.org/oss-sec/2026/q2/332
PS. I'm not dunking on Rust. It's just that... starting over from scratch has its hidden costs.
@lcamtuf Amusingly, I recently did some work in Rust and wanted safe file operations that avoided race conditions. I couldn't find anything good and wrote my own opinionated helper.
Though, a large part of it is that O_TMPFILE is awesome and underused.
-
The coreutils Rust rewrite story is pretty funny.
Coreutils are tools like rm, mv, mkdir, etc. Unlike binutils, this isn't a fertile ground for memory safety bugs. But, the rewrite was completed, and in the spirit of progress, Canonical decided to switch.
But do you know what coreutils are a fertile ground for? Race conditions around file creation, deletion, permission setting, and so on. The original code accounted for decades of hard-learned lessons in that space. The Rust rewrite did not:
https://seclists.org/oss-sec/2026/q2/332
PS. I'm not dunking on Rust. It's just that... starting over from scratch has its hidden costs.
@lcamtuf@infosec.exchange Also quite few are noticeably fails in implementing POSIX, which makes me wonder if they’re only caring about coreutils testsuite and
--help/help2manoutput.Like CVE-2026-35367 (
nohup(1)permissions) as Colin Funk noted, but also CVE-2026-35369 (kill -1), CVE-2026-35370 & CVE-2026-35371 (real vs. effective inid(1)), and CVE-2026-35379 (wrong character classes intr(1)) -
The coreutils Rust rewrite story is pretty funny.
Coreutils are tools like rm, mv, mkdir, etc. Unlike binutils, this isn't a fertile ground for memory safety bugs. But, the rewrite was completed, and in the spirit of progress, Canonical decided to switch.
But do you know what coreutils are a fertile ground for? Race conditions around file creation, deletion, permission setting, and so on. The original code accounted for decades of hard-learned lessons in that space. The Rust rewrite did not:
https://seclists.org/oss-sec/2026/q2/332
PS. I'm not dunking on Rust. It's just that... starting over from scratch has its hidden costs.
@lcamtuf I've heard a lot of funny stories like this in previous years. Like for example a startup trying to rewrite the TCP stack by their own from scratch because they can do it more efficient.
Soon they learned how a real environment, or better said, the real life really is. -
The coreutils Rust rewrite story is pretty funny.
Coreutils are tools like rm, mv, mkdir, etc. Unlike binutils, this isn't a fertile ground for memory safety bugs. But, the rewrite was completed, and in the spirit of progress, Canonical decided to switch.
But do you know what coreutils are a fertile ground for? Race conditions around file creation, deletion, permission setting, and so on. The original code accounted for decades of hard-learned lessons in that space. The Rust rewrite did not:
https://seclists.org/oss-sec/2026/q2/332
PS. I'm not dunking on Rust. It's just that... starting over from scratch has its hidden costs.
@lcamtuf yeah it's frustrating because in some sense we all had the opportunity to learn this lesson, a long time ago
we remember when we were kids, after Netscape went bankrupt trying to re-write their software from scratch, there were some good essays analyzing what went wrong and advocating for refactoring instead so as not to lose the knowledge that's in the code
and then there's the ATC system
like... there's so many past instances to learn from
-
The coreutils Rust rewrite story is pretty funny.
Coreutils are tools like rm, mv, mkdir, etc. Unlike binutils, this isn't a fertile ground for memory safety bugs. But, the rewrite was completed, and in the spirit of progress, Canonical decided to switch.
But do you know what coreutils are a fertile ground for? Race conditions around file creation, deletion, permission setting, and so on. The original code accounted for decades of hard-learned lessons in that space. The Rust rewrite did not:
https://seclists.org/oss-sec/2026/q2/332
PS. I'm not dunking on Rust. It's just that... starting over from scratch has its hidden costs.
@lcamtuf I mean, not a single one of the issues were memory-safety-related, which is noteworthy, cause GNU/BSD coreutils still regularly get memory-related issues
-
The coreutils Rust rewrite story is pretty funny.
Coreutils are tools like rm, mv, mkdir, etc. Unlike binutils, this isn't a fertile ground for memory safety bugs. But, the rewrite was completed, and in the spirit of progress, Canonical decided to switch.
But do you know what coreutils are a fertile ground for? Race conditions around file creation, deletion, permission setting, and so on. The original code accounted for decades of hard-learned lessons in that space. The Rust rewrite did not:
https://seclists.org/oss-sec/2026/q2/332
PS. I'm not dunking on Rust. It's just that... starting over from scratch has its hidden costs.
@lcamtuf Dang, that is a wild ride of a thread.
And it kinda lines up with my experiences as well-- coreutils is battle tested and a load bearing feature of Linux.
Uutils is just too new to get all of the behavior exactly the same. I've tested it on my nix machine in the past, and alothough I never pushed uutils quite as far as it could have gone in order to discover any of these bugs, I kind of shudder to think what would have happened if I had.
Very interesting to think that the concept of C isn't exactly bad-- but it just needs a long time to mature and get it right, just like any program. The fact that the Rust compiler prevents you from making memory errors doesn't also prevent you from misunderstanding CPU clocks or buffer overflows or race conditions and other low level stuff.
-
@m33
I discovered at Google a tremendous laziness and lack of rigor because "well if it doesn't work or has problems we can roll it back." I came to think of it as The Google Principle and it can be more easily written as:The amount of care and thought that goes into a software change is proportional to the perceived difficulty of pushing that change into production.
@ChuckMcManis @m33 @sten @lcamtuf on the flip side, if you have good discipline around handling change on a continuous basis and operational agility, you can more easily incorporate Werner Vogels' aphorism "everything fails, all the time, plan accordingly"
does that lead to lazy / negligent engineering? maybe?? If the architecture is such that I don't have to care as much about my component failing, maybe we need different metrics to incentivize quality other than "it went down and people got fired"
-
@lcamtuf yeah it's frustrating because in some sense we all had the opportunity to learn this lesson, a long time ago
we remember when we were kids, after Netscape went bankrupt trying to re-write their software from scratch, there were some good essays analyzing what went wrong and advocating for refactoring instead so as not to lose the knowledge that's in the code
and then there's the ATC system
like... there's so many past instances to learn from
@lcamtuf and then there's... well, there's a persistent feeling that starting over without regard for the past will make things better, rather than just repeating the same fundamental mistake that happened the first time
we've felt it too. it's a powerful pull.
we wrote a bit about that feeling, a while back https://irenes.space/leaves/2024-09-29-technology-community-idealism
-
@darkuncle @lcamtuf
During my tenure at Google I was astonished at how many engineers would clearly admit they didn't understand why something was the way it was, so they rewrote it. This *repeatedly* bit them in the ass.@ChuckMcManis I actually find questioning the why behind something to be important. In your experience at Google, did the devs rewriting things have _access_ to the documentation as to why something was done? Was it like disbelief of the stated facts or were there holes in the notetaking about the reasoning?
-
@ChuckMcManis I actually find questioning the why behind something to be important. In your experience at Google, did the devs rewriting things have _access_ to the documentation as to why something was done? Was it like disbelief of the stated facts or were there holes in the notetaking about the reasoning?
@ChuckMcManis addendum: I didn't mean for this to be a "well, actually" statement; I'm not pushing back against your statements, only curious about your experience.
-
@lcamtuf mumble, mumble, Chesterson's Fence, mumble, mumble
@ChuckMcManis @lcamtuf now why the hell, after all these years, have I not heard of Chesterton’s Fence? Is this what I missed by not learning my trade at a uni? Kinda rhetorical, but I _still_ feel I must’ve missed something other than the debt…
Anyway, thank you. It’s going to provide a nice two-worder when reviewing prs by newbies - and a lot of oldbies and, lest l forget, their sloppy chums.
-
The coreutils Rust rewrite story is pretty funny.
Coreutils are tools like rm, mv, mkdir, etc. Unlike binutils, this isn't a fertile ground for memory safety bugs. But, the rewrite was completed, and in the spirit of progress, Canonical decided to switch.
But do you know what coreutils are a fertile ground for? Race conditions around file creation, deletion, permission setting, and so on. The original code accounted for decades of hard-learned lessons in that space. The Rust rewrite did not:
https://seclists.org/oss-sec/2026/q2/332
PS. I'm not dunking on Rust. It's just that... starting over from scratch has its hidden costs.
@lcamtuf "The lesson of history is that no one learns."
-
@lcamtuf and then there's... well, there's a persistent feeling that starting over without regard for the past will make things better, rather than just repeating the same fundamental mistake that happened the first time
we've felt it too. it's a powerful pull.
we wrote a bit about that feeling, a while back https://irenes.space/leaves/2024-09-29-technology-community-idealism
@ireneista @lcamtuf
I guess that could work if you really investigate all the fundamental mistakes, as well as the regular bugs/pitfalls, from the first time and try your best to avoid them.Assuming that "it was written in a less safe language" was the only or even most important issue is.. not that useful
-
@ireneista @lcamtuf
I guess that could work if you really investigate all the fundamental mistakes, as well as the regular bugs/pitfalls, from the first time and try your best to avoid them.Assuming that "it was written in a less safe language" was the only or even most important issue is.. not that useful
@Doomed_Daniel @lcamtuf yeah, exactly
-
It’s frustrating that POSIX took decades to get APIs that weren’t intrinsically racy, but then higher-level languages that post dated the improved ones implemented equivalents of the old racy APIs. C++ was annoying, they waited until pretty much every platform that supported C++ and had a filesystem implemented the newer APIs and then standardised the filesystem TS with racy ones. I believe Rust is similar, but at least it has cap-std which implements the non-racy versions as an alternative standard library.
@david_chisnall @lcamtuf Try to write to C++ ‚cout‘ concurrently. Complete clown fiesta!
-
@lcamtuf and then there's... well, there's a persistent feeling that starting over without regard for the past will make things better, rather than just repeating the same fundamental mistake that happened the first time
we've felt it too. it's a powerful pull.
we wrote a bit about that feeling, a while back https://irenes.space/leaves/2024-09-29-technology-community-idealism
@ireneista @lcamtuf I would also assume that there is simply a tipping point in regards to the scale of a project, where a rewrite of the whole thing at once just doesn't make sense anymore.
For very mall things, it obvious makes sense, because it's easy to hold the entire architecture and important fixes in a single persons head (and thus also documenting them would be doable), but there comes a point where even a large team can't do that in a cooperative manner anymore.
