This "careful" "AI Safety" company that just accidentally leaked its entire source code to the world is the one that African governments are entering into agreements with to include in infrastructures from health care to god knows what.
-
This "careful" "AI Safety" company that just accidentally leaked its entire source code to the world is the one that African governments are entering into agreements with to include in infrastructures from health care to god knows what.
These are the products people have to use to make sure that they don't get dinged in their performance reviews for "not using AI."
These are the products teachers have to use in schools so that "students aren't left behind."
https://www.theguardian.com/technology/2026/apr/01/anthropic-claudes-code-leaks-ai
-
This "careful" "AI Safety" company that just accidentally leaked its entire source code to the world is the one that African governments are entering into agreements with to include in infrastructures from health care to god knows what.
These are the products people have to use to make sure that they don't get dinged in their performance reviews for "not using AI."
These are the products teachers have to use in schools so that "students aren't left behind."
https://www.theguardian.com/technology/2026/apr/01/anthropic-claudes-code-leaks-ai
I appreciated this article by @mttaggart
infosec.exchange.I get the temptation especially in this world we're all living in where you have to produce something super fast all the time.
But my question is, what are people's arguments for how functioning software can be created with these tools?
What about new architectures, new ways of thinking, new programming languages, etc? Who will create those?
-
I appreciated this article by @mttaggart
infosec.exchange.I get the temptation especially in this world we're all living in where you have to produce something super fast all the time.
But my question is, what are people's arguments for how functioning software can be created with these tools?
What about new architectures, new ways of thinking, new programming languages, etc? Who will create those?
I'm not even talking about the data stealing, exploitation, environmental pillaging, pollution, environmental racism etc.
I'm talking about the way people use the tools. Like what do advocates of using these tools say will happen to software engineering in the future? That it just won't need to exist because everyone will be able to create software using these tools?
-
I'm not even talking about the data stealing, exploitation, environmental pillaging, pollution, environmental racism etc.
I'm talking about the way people use the tools. Like what do advocates of using these tools say will happen to software engineering in the future? That it just won't need to exist because everyone will be able to create software using these tools?
That it will just take a different form, which is fine?
-
I'm not even talking about the data stealing, exploitation, environmental pillaging, pollution, environmental racism etc.
I'm talking about the way people use the tools. Like what do advocates of using these tools say will happen to software engineering in the future? That it just won't need to exist because everyone will be able to create software using these tools?
@timnitGebru You put it into words better than me: https://mastodon.social/@cr1901/115844213832136867
-
That it will just take a different form, which is fine?
@timnitGebru Yes. To a large degree, I think it's fine.
And the old forms will still be there in a lot of cases and contexts. And, if we build the future well, we won't put hard barriers to digging in and finding out what's going on. If we build it poorly and let platform rentiership win, that's a big problem loomng.
-
That it will just take a different form, which is fine?
@timnitGebru I think this is relevant to these questions, albeit handles them on a different level:
https://freakonometrics.hypotheses.org/89367> Someone still has to reread, compare, test, contextualize, and sometimes rewrite. And if no one seriously takes on that work, the cost does not disappear. It reappears later in the form of errors, urgent fixes, loss of trust, and eventually litigation. What is presented as a productivity gain is often just an accounting displacement.
-
I'm not even talking about the data stealing, exploitation, environmental pillaging, pollution, environmental racism etc.
I'm talking about the way people use the tools. Like what do advocates of using these tools say will happen to software engineering in the future? That it just won't need to exist because everyone will be able to create software using these tools?
@timnitGebru I really don’t understand why they find this even remotely appealing. Are they really convinced that software has no differentiated value? Quality and *correctness* are luxuries? Everything will just be a uniform beige paste with the same beige bugs.
-
I'm not even talking about the data stealing, exploitation, environmental pillaging, pollution, environmental racism etc.
I'm talking about the way people use the tools. Like what do advocates of using these tools say will happen to software engineering in the future? That it just won't need to exist because everyone will be able to create software using these tools?
@timnitGebru I am refactoring a mid 00s era C++ code base and so far, because of the subtleties of its architecture, I have found it better to do some of the "lame" refactorings by hand because of the risk an "AI" agent would misread things completely and make breaking changes. As this work proceeds the scope for GenAI is getting narrower and narrower, like, substitute include guards in C/C++ with pragma once, or specific changes suggested in John Lakos et al's EMC++S book.
-
I'm not even talking about the data stealing, exploitation, environmental pillaging, pollution, environmental racism etc.
I'm talking about the way people use the tools. Like what do advocates of using these tools say will happen to software engineering in the future? That it just won't need to exist because everyone will be able to create software using these tools?
@timnitGebru EMC++S: Embracing Modern C++ Safely. My appetite for actually using GenAI is wearing thin after the severe information security risk Claude Code and other frontends are known to pose, after the leak <48 hours ago. LLMs have suggested regular expressions to me, but their role has been pretty limited to that of a error prone natural language search processor for me. This suggests a far lower economic point of inflexion for GenAI driven advantage than that promoted for it.
-
@timnitGebru I think this is relevant to these questions, albeit handles them on a different level:
https://freakonometrics.hypotheses.org/89367> Someone still has to reread, compare, test, contextualize, and sometimes rewrite. And if no one seriously takes on that work, the cost does not disappear. It reappears later in the form of errors, urgent fixes, loss of trust, and eventually litigation. What is presented as a productivity gain is often just an accounting displacement.
@rysiek Great article.
-
@rysiek Great article.
@timnitGebru it really is.
And boy does the Claude Code leaked codebase support that assessment. Have you seen @jonny 's thread on this? If not:
https://neuromatch.social/@jonny/116324676116121930 -
@timnitGebru it really is.
And boy does the Claude Code leaked codebase support that assessment. Have you seen @jonny 's thread on this? If not:
https://neuromatch.social/@jonny/116324676116121930@timnitGebru the whole thing is great, but somewhere down the thread there are truly astonishing gems like:
> So the reason that Claude code is capable of outputting valid json is because if the prompt text suggests it should be JSON then it enters a special loop in the main query engine that just validates it against JSON schema for JSON and then feeds the data with the error message back into itself until it is valid JSON or a retry limit is reached.
Thousand monkeys, thousand typewriters…
-
@timnitGebru EMC++S: Embracing Modern C++ Safely. My appetite for actually using GenAI is wearing thin after the severe information security risk Claude Code and other frontends are known to pose, after the leak <48 hours ago. LLMs have suggested regular expressions to me, but their role has been pretty limited to that of a error prone natural language search processor for me. This suggests a far lower economic point of inflexion for GenAI driven advantage than that promoted for it.
@timnitGebru Also, a lot of the FreeBSD related work I've been doing lately hasn't been writing software itself in anger, but hardware qualification: physically plugging hardware together, usually network adapters, switches, and routers, and evaluating compatibility. Using agents for any of this, whilst possible, would be like putting a hat on a hat, to borrow an expression from Seth MacFarlane in Family Guy. The human factor reigns supreme because of ISO OSI Layer 1.
-
@timnitGebru the whole thing is great, but somewhere down the thread there are truly astonishing gems like:
> So the reason that Claude code is capable of outputting valid json is because if the prompt text suggests it should be JSON then it enters a special loop in the main query engine that just validates it against JSON schema for JSON and then feeds the data with the error message back into itself until it is valid JSON or a retry limit is reached.
Thousand monkeys, thousand typewriters…
@timnitGebru of course it makes total sense for Claude Code to waste developer tokens like that, since Anthropic charges per token…
-
I appreciated this article by @mttaggart
infosec.exchange.I get the temptation especially in this world we're all living in where you have to produce something super fast all the time.
But my question is, what are people's arguments for how functioning software can be created with these tools?
What about new architectures, new ways of thinking, new programming languages, etc? Who will create those?
@timnitGebru that blogpost strikes me as incredibly irresponsible
The legalistic use of the word "works" - the post itself includes the keyphrase "works with caveats"! - and that otherwise reasonable conclusion that becomes absolutely heinous anywhere that isn't a vacuum. Suggesting people need to be more accommodating towards LLM users is a joke when this is the cohort attempting to force their (by the authors' recognition horrifically joyless to use) toys onto and into everyone else's life.
-
@timnitGebru that blogpost strikes me as incredibly irresponsible
The legalistic use of the word "works" - the post itself includes the keyphrase "works with caveats"! - and that otherwise reasonable conclusion that becomes absolutely heinous anywhere that isn't a vacuum. Suggesting people need to be more accommodating towards LLM users is a joke when this is the cohort attempting to force their (by the authors' recognition horrifically joyless to use) toys onto and into everyone else's life.
@timnitGebru In a perfect world I'd accept people that love their codegen chatbots as no different from people that prefer the command line or tabs over spaces!
But we're not in that world and they're actively forcing their products on everyone else and posts like these reek of someone that has the privilege of not having that be done to them.
-
@timnitGebru it really is.
And boy does the Claude Code leaked codebase support that assessment. Have you seen @jonny 's thread on this? If not:
https://neuromatch.social/@jonny/116324676116121930 -
@timnitGebru of course it makes total sense for Claude Code to waste developer tokens like that, since Anthropic charges per token…
@rysiek Literally the questions of "what if computer science was no longer about figuring out the most efficient way to do X but the brute force way to do X"?
-
@timnitGebru the whole thing is great, but somewhere down the thread there are truly astonishing gems like:
> So the reason that Claude code is capable of outputting valid json is because if the prompt text suggests it should be JSON then it enters a special loop in the main query engine that just validates it against JSON schema for JSON and then feeds the data with the error message back into itself until it is valid JSON or a retry limit is reached.
Thousand monkeys, thousand typewriters…
@rysiek @timnitGebru The illusion of progress, indeed! I plan to do my initial experiments with Gemini as it is being massively subsidised at the open API gateway level via Opencode.AI, as opposed to using monthly subscriptions for the now arguably massively discredited Claude Code. That's if I even get around to it. So far just using project-wide find/grep/sed magic is working just fine for me, and traditional clang-tidy abstract syntax tree (AST) based refactoring is closer in grasp.
