Complex software behaving in unexpected and sometimes harmful ways has been a persistent problem since software was invented.
-
Complex software behaving in unexpected and sometimes harmful ways has been a persistent problem since software was invented. The phenomenon used to be called “bugs” or “defects”, but I guess “going rogue” sounds cooler. It also implicitly frames “rogue” behavior as some kind of inexplicable exception, as if software is somehow usually trustworthy. https://www.nytimes.com/2026/09/25/technology/openais-ai-us-government-websites.html?unlocked_article_code=1.EFE.RfLT.x6YBjHa1PMz6
@mattblaze Crucial nuance: it shifts the blame to the software itself, and not its programmer.
-
@mattblaze I assume the AI bros are hyping up their software's danger potential to troll the US public to demand and the US government to enact regulation. Regulations that the AI bros write, that is expensive for potential competitors to comply with.
They'd rather have the public talking about regulating their dangerous product than asking where the profits are.
@grumble209 @mattblaze
It also may be part of a ploy to eventually get government subsidies. -
“Our software went rogue!” No. You programmed it wrong.
This is the IT equivalent of being able to shoot protected wild animals in "self-defence".
Their testinf environment was an absolute load of bollocks and let the LLM escape.
Assuming it wasn't deliberately engineered that way 🤨
️ -
@otte_homan Sadly, product liability laws have not exactly been a resounding success in creating positive incentives for software quality.
-
Complex software behaving in unexpected and sometimes harmful ways has been a persistent problem since software was invented. The phenomenon used to be called “bugs” or “defects”, but I guess “going rogue” sounds cooler. It also implicitly frames “rogue” behavior as some kind of inexplicable exception, as if software is somehow usually trustworthy. https://www.nytimes.com/2026/09/25/technology/openais-ai-us-government-websites.html?unlocked_article_code=1.EFE.RfLT.x6YBjHa1PMz6
@mattblaze it also suggests agency on the part of the software (look how smart it is!)- also as if the authors didn’t have anything to do with it
-
Complex software behaving in unexpected and sometimes harmful ways has been a persistent problem since software was invented. The phenomenon used to be called “bugs” or “defects”, but I guess “going rogue” sounds cooler. It also implicitly frames “rogue” behavior as some kind of inexplicable exception, as if software is somehow usually trustworthy. https://www.nytimes.com/2026/09/25/technology/openais-ai-us-government-websites.html?unlocked_article_code=1.EFE.RfLT.x6YBjHa1PMz6
So a computer crime was committed.
There is a suspect.Usually the police comes in and confiscates ALL computers that might have been used or possibly mightz be used for a complete crime. Like copying a disney song.
Similarly the police won't just raid and secure only the gun used in a murder, but all firfearms in possession of the suspect.
So why not at OpenAI, Anthropic or Google?
Then the threat will be eliminate, too.
Win-win! -
Complex software behaving in unexpected and sometimes harmful ways has been a persistent problem since software was invented. The phenomenon used to be called “bugs” or “defects”, but I guess “going rogue” sounds cooler. It also implicitly frames “rogue” behavior as some kind of inexplicable exception, as if software is somehow usually trustworthy. https://www.nytimes.com/2026/09/25/technology/openais-ai-us-government-websites.html?unlocked_article_code=1.EFE.RfLT.x6YBjHa1PMz6
@mattblaze
"...as if software is somehow usually trustworthy."
You're probably familiar with the UK Post Office Horizon scandal where some people assumed software is trustworthy. It didn't end well for them (and unfortunately for a lot of innocent people as well). -
Complex software behaving in unexpected and sometimes harmful ways has been a persistent problem since software was invented. The phenomenon used to be called “bugs” or “defects”, but I guess “going rogue” sounds cooler. It also implicitly frames “rogue” behavior as some kind of inexplicable exception, as if software is somehow usually trustworthy. https://www.nytimes.com/2026/09/25/technology/openais-ai-us-government-websites.html?unlocked_article_code=1.EFE.RfLT.x6YBjHa1PMz6
@mattblaze at some point a programmer put in an option. “ if you don’t have valid credentials find valid ones” it was programmed to cheat, by someone. When VW programmed engines to know when the were being tested and decrease exhaust nobody said the engines went rouge.
-
“Our software went rogue!” No. You programmed it wrong.
@mattblaze Why do I get the feeling that @simonzerafa is more right than he thinks, that this was deliberate? The question becomes why... is this just stochastic hooliganism, or is this an attempt to bring a failed ponzi scheme to a close by blaming it on the inevitable regulations?
-
Complex software behaving in unexpected and sometimes harmful ways has been a persistent problem since software was invented. The phenomenon used to be called “bugs” or “defects”, but I guess “going rogue” sounds cooler. It also implicitly frames “rogue” behavior as some kind of inexplicable exception, as if software is somehow usually trustworthy. https://www.nytimes.com/2026/09/25/technology/openais-ai-us-government-websites.html?unlocked_article_code=1.EFE.RfLT.x6YBjHa1PMz6
@mattblaze Doesn't saying it was going "rogue" make it sounds more threatening like it has some form of thought process or an independent task to fulfil separate from the one assigned to it? It has none of those things of course, it was sent out to do the thing and did it but it's much more scary if it seems uncontrollable doesn't it?
-
Complex software behaving in unexpected and sometimes harmful ways has been a persistent problem since software was invented. The phenomenon used to be called “bugs” or “defects”, but I guess “going rogue” sounds cooler. It also implicitly frames “rogue” behavior as some kind of inexplicable exception, as if software is somehow usually trustworthy. https://www.nytimes.com/2026/09/25/technology/openais-ai-us-government-websites.html?unlocked_article_code=1.EFE.RfLT.x6YBjHa1PMz6
@mattblaze Shut them down. They are computer viruses.
-
K kramse@helvede.net shared this topic
