The coreutils Rust rewrite story is pretty funny.
-
@lcamtuf and then there's... well, there's a persistent feeling that starting over without regard for the past will make things better, rather than just repeating the same fundamental mistake that happened the first time
we've felt it too. it's a powerful pull.
we wrote a bit about that feeling, a while back https://irenes.space/leaves/2024-09-29-technology-community-idealism
@ireneista @lcamtuf I would also assume that there is simply a tipping point in regards to the scale of a project, where a rewrite of the whole thing at once just doesn't make sense anymore.
For very mall things, it obvious makes sense, because it's easy to hold the entire architecture and important fixes in a single persons head (and thus also documenting them would be doable), but there comes a point where even a large team can't do that in a cooperative manner anymore.
-
@ireneista @lcamtuf I would also assume that there is simply a tipping point in regards to the scale of a project, where a rewrite of the whole thing at once just doesn't make sense anymore.
For very mall things, it obvious makes sense, because it's easy to hold the entire architecture and important fixes in a single persons head (and thus also documenting them would be doable), but there comes a point where even a large team can't do that in a cooperative manner anymore.
@ireneista @lcamtuf Maybe it's related to the tipping points in "groups of humans" sizes, where there's a point where communication becomes less efficient and a probably also a bottleneck.
-
@lcamtuf and it's very worth remembering that while the design of rust _does_ prevent many bugs, it's not a get-out-of-bugs-free card. there are many ways to write code wrong, not just memory safety issues!
-
@lcamtuf i do find that the crates dedicated to atomic file handling and temp files, in the interest of providing a uniform platform interface aren't as good as what's reachable in c.
it's not a fault of the rust language per se, but writing a safe interface at that level isn't easy, so it makes sense (and is in some sense a better default) to have high level, platform neutral access here.
Could you elaborate on this? What exactly is lacking in e.g. the tempfile crate? If it were for Linux only, in what ways would it be better?
-
The coreutils Rust rewrite story is pretty funny.
Coreutils are tools like rm, mv, mkdir, etc. Unlike binutils, this isn't a fertile ground for memory safety bugs. But, the rewrite was completed, and in the spirit of progress, Canonical decided to switch.
But do you know what coreutils are a fertile ground for? Race conditions around file creation, deletion, permission setting, and so on. The original code accounted for decades of hard-learned lessons in that space. The Rust rewrite did not:
https://seclists.org/oss-sec/2026/q2/332
PS. I'm not dunking on Rust. It's just that... starting over from scratch has its hidden costs.
@lcamtuf I legitimately wonder what it is about Rust that inspires people to start questionable porting projects in the first place. Like, who asked for coreutils in Rust?
-
@lcamtuf See this all the time - people storm in trying to change things before trying to understand how the current things work. People who don't learn from what's been done before. Society doesn't progress from efforts like theirs. You only make progress by learning from and building on top of what came before.
Even if you do tear down what came before and replace it entirely, you still need to understand how the thing you're replacing works, or else your replacement will be worse because you'll make the same mistakes your predecessors did.
Those who fail to learn from history are doomed to repeat it, after all.
-
-
@ChuckMcManis I actually find questioning the why behind something to be important. In your experience at Google, did the devs rewriting things have _access_ to the documentation as to why something was done? Was it like disbelief of the stated facts or were there holes in the notetaking about the reasoning?
@josh To be clear, all the source code, a wiki of discussions, etc was available to everyone who wrote code at the time. I recall pushing back one such occasion and asking if the project lead knew *why* the current code did what it did, and they stated, and I quote because it stuck with me, "It doesn't matter why, because I've rewritten all of it and I know why I wrote what I did."
To understand that you have to understand Google dev culture at the time. 1/2
-
Even if you do tear down what came before and replace it entirely, you still need to understand how the thing you're replacing works, or else your replacement will be worse because you'll make the same mistakes your predecessors did.
Those who fail to learn from history are doomed to repeat it, after all.
@argv_minus_one @lcamtuf my point exactly
-
@lcamtuf yeah it's frustrating because in some sense we all had the opportunity to learn this lesson, a long time ago
we remember when we were kids, after Netscape went bankrupt trying to re-write their software from scratch, there were some good essays analyzing what went wrong and advocating for refactoring instead so as not to lose the knowledge that's in the code
and then there's the ATC system
like... there's so many past instances to learn from
"Netscape went bankrupt trying to re-write their software from scratch"
It is also why Microsoft Edge went from something written from scratch to be a fork of chromium. The story is the same and even more it is about the similar product. Plus it is a recent example of the whole starting from scratch issues.
-
@ireneista @lcamtuf
Additional fun thought: I can imagine they avoided looking at the GNU coreutils C implementation because they are using MIT license instead of GPL. -
@josh To be clear, all the source code, a wiki of discussions, etc was available to everyone who wrote code at the time. I recall pushing back one such occasion and asking if the project lead knew *why* the current code did what it did, and they stated, and I quote because it stuck with me, "It doesn't matter why, because I've rewritten all of it and I know why I wrote what I did."
To understand that you have to understand Google dev culture at the time. 1/2
Dev culture was one long series of "dick measurements" as one engineer put it. That was because of how Google evaluated engineers and how reviews got written. How that emerged in practice was that saying "I don't know" was like saying "fire me now" to a lot of these kids. So they couldn't introspect at all in front of others. I was not popular (as you might guess) for asking "rude" questions.
But I also didn't care if Google chose to fire me so there was that. 2/2 -
@ireneista @lcamtuf I think in this case it would've been possible to properly rewrite hem one by one, but the metric appears to be more a quantity-over-quality "lets get this out of the door quickly", i.e. the incentives weren't placed right.
-
@lcamtuf I've heard a lot of funny stories like this in previous years. Like for example a startup trying to rewrite the TCP stack by their own from scratch because they can do it more efficient.
Soon they learned how a real environment, or better said, the real life really is. -
The coreutils Rust rewrite story is pretty funny.
Coreutils are tools like rm, mv, mkdir, etc. Unlike binutils, this isn't a fertile ground for memory safety bugs. But, the rewrite was completed, and in the spirit of progress, Canonical decided to switch.
But do you know what coreutils are a fertile ground for? Race conditions around file creation, deletion, permission setting, and so on. The original code accounted for decades of hard-learned lessons in that space. The Rust rewrite did not:
https://seclists.org/oss-sec/2026/q2/332
PS. I'm not dunking on Rust. It's just that... starting over from scratch has its hidden costs.
isn't a fertile ground for memory safety bugs
Mm. Another write-up, admittedly on a pro-Rust site, collected the recent GNU coreutils CVEs and they do seem to still be having memory safety bugs even after X decades of development. Are these as bad as TOCTOU bugs? Probably not, but I'm not sure "start over from scratch" comes off as poorly with these stats. https://corrode.dev/blog/bugs-rust-wont-catch/#what-rust-did-prevent
-
@ireneista@adhd.irenes.space @lcamtuf@infosec.exchange @Doomed_Daniel@mastodon.gamedev.place Well there's always the ability for clean room implementation, no?
-
@ireneista@adhd.irenes.space @lcamtuf@infosec.exchange @Doomed_Daniel@mastodon.gamedev.place Well there's always the ability for clean room implementation, no?
@lcamtuf@infosec.exchange @Doomed_Daniel@mastodon.gamedev.place @ireneista@adhd.irenes.space I will admit it does sound extremely stupid to cleanroom implement coreutils but at the same time I feel like that could have mitigated some of the issue here.
-
The coreutils Rust rewrite story is pretty funny.
Coreutils are tools like rm, mv, mkdir, etc. Unlike binutils, this isn't a fertile ground for memory safety bugs. But, the rewrite was completed, and in the spirit of progress, Canonical decided to switch.
But do you know what coreutils are a fertile ground for? Race conditions around file creation, deletion, permission setting, and so on. The original code accounted for decades of hard-learned lessons in that space. The Rust rewrite did not:
https://seclists.org/oss-sec/2026/q2/332
PS. I'm not dunking on Rust. It's just that... starting over from scratch has its hidden costs.
@lcamtuf they did a writeup here https://corrode.dev/blog/bugs-rust-wont-catch/#what-rust-did-prevent
to be fair, the original uutils may not be a hotbed of memory safety bugs but the linked section cites a bunch
-
@Doomed_Daniel@mastodon.gamedev.place @lcamtuf@infosec.exchange @ireneista@adhd.irenes.space I think you mistook musl for BusyBox. I did the same thing in my head. (musl is an MIT-licensed C standard library implementation) It appears BusyBox (alternative to gnu coreutils) is also licensed under GPLv2, unfortunately.
-
@david_chisnall @lcamtuf Try to write to C++ ‚cout‘ concurrently. Complete clown fiesta!
using std::cout concurrently does not cause data races (no UB). If youvwant to get output readably together use std::osyncstream wrapper around the global object or any other shared ostream object.
