Complex software behaving in unexpected and sometimes harmful ways has been a persistent problem since software was invented.
-
Complex software behaving in unexpected and sometimes harmful ways has been a persistent problem since software was invented. The phenomenon used to be called “bugs” or “defects”, but I guess “going rogue” sounds cooler. It also implicitly frames “rogue” behavior as some kind of inexplicable exception, as if software is somehow usually trustworthy. https://www.nytimes.com/2026/09/25/technology/openais-ai-us-government-websites.html?unlocked_article_code=1.EFE.RfLT.x6YBjHa1PMz6
@mattblaze
I totally agree, one of the biggest and most successful bits of the lol marketing scan is the choice of terms indicating intelligence or some kind of life in the machine.
it's not even a bug, LLMs are intentionally non-deterministic and so random outputs should be expected.
If sometime shoots a gun into the air and accidentally killed someone, the bullet didn't 'go rogue'. The shooter should be prosecuted. Same principal applies here imo. -
“Our software went rogue!” No. You programmed it wrong.
@mattblaze I assume the AI bros are hyping up their software's danger potential to troll the US public to demand and the US government to enact regulation. Regulations that the AI bros write, that is expensive for potential competitors to comply with.
They'd rather have the public talking about regulating their dangerous product than asking where the profits are.
-
Complex software behaving in unexpected and sometimes harmful ways has been a persistent problem since software was invented. The phenomenon used to be called “bugs” or “defects”, but I guess “going rogue” sounds cooler. It also implicitly frames “rogue” behavior as some kind of inexplicable exception, as if software is somehow usually trustworthy. https://www.nytimes.com/2026/09/25/technology/openais-ai-us-government-websites.html?unlocked_article_code=1.EFE.RfLT.x6YBjHa1PMz6
@mattblaze "it's not our fault! It went rogue!"
Incompetent.
-
@mattblaze I assume the AI bros are hyping up their software's danger potential to troll the US public to demand and the US government to enact regulation. Regulations that the AI bros write, that is expensive for potential competitors to comply with.
They'd rather have the public talking about regulating their dangerous product than asking where the profits are.
@grumble209 I’m not sure exactly what their game is here, but large incumbents LOVE gatekeeping regulation.
-
“Our software went rogue!” No. You programmed it wrong.
@mattblaze rude, stop attacking my plausible deniability
-
Complex software behaving in unexpected and sometimes harmful ways has been a persistent problem since software was invented. The phenomenon used to be called “bugs” or “defects”, but I guess “going rogue” sounds cooler. It also implicitly frames “rogue” behavior as some kind of inexplicable exception, as if software is somehow usually trustworthy. https://www.nytimes.com/2026/09/25/technology/openais-ai-us-government-websites.html?unlocked_article_code=1.EFE.RfLT.x6YBjHa1PMz6
@mattblaze it wasn't programmed wrong, it's kind of designed to do this. It's either: they are brazenly drumming up issues to boost stock prices or it's a consequence of firing/not hiring in the first place people who know how to manage these situations (so they can boost stock prices).
Or it's just an attempt to boost stock prices. It might also be to do with stock prices.
-
“Our software went rogue!” No. You programmed it wrong.
-
@mattblaze rude, stop attacking my plausible deniability
plausible denAIability
-
@mattblaze rude, stop attacking my plausible deniability
The best side effect of the AI is now it's possible to blame it for almost every human error.
«Sorry, officer, not my fault, but the AI pushed the car at 84 mph!»
-
Complex software behaving in unexpected and sometimes harmful ways has been a persistent problem since software was invented. The phenomenon used to be called “bugs” or “defects”, but I guess “going rogue” sounds cooler. It also implicitly frames “rogue” behavior as some kind of inexplicable exception, as if software is somehow usually trustworthy. https://www.nytimes.com/2026/09/25/technology/openais-ai-us-government-websites.html?unlocked_article_code=1.EFE.RfLT.x6YBjHa1PMz6
@mattblaze
" It's not a bug, it's a feature! " . -
“Our software went rogue!” No. You programmed it wrong.
@mattblaze
When is Judgement Day ? (Financial reports indicating the fraud).
Supposed to be in October.
All this shit talk just to avoid it. -
@mattblaze I assume the AI bros are hyping up their software's danger potential to troll the US public to demand and the US government to enact regulation. Regulations that the AI bros write, that is expensive for potential competitors to comply with.
They'd rather have the public talking about regulating their dangerous product than asking where the profits are.
@grumble209 @mattblaze Another angle on this is to try and avoid culpability for the crimes they let their LLM A.I. agents do. Like, the Hugging Face hack that OpenAI did had a lot of people thinking "Wait - isn't hacking already illegal? It's a felony, isn't it? Why wasn't someone at OpenAI charged with that crime?", a lot of people claimed "Well, the operator didn't tell the LLM to commit the crime, nor did the trainers of the model, so who do we hold responsible?" as why it wouldn't happen despite the company announcing they had something at their company commit an actual existing felony.
Even in the best take, negligence feels like it should be the person most in charge of the test to be charged with the hacking, and/or the network monitors that didn't realize how much network traffic was spent to hack Hugging Face from OpenAI.
-
@grumble209 @mattblaze Another angle on this is to try and avoid culpability for the crimes they let their LLM A.I. agents do. Like, the Hugging Face hack that OpenAI did had a lot of people thinking "Wait - isn't hacking already illegal? It's a felony, isn't it? Why wasn't someone at OpenAI charged with that crime?", a lot of people claimed "Well, the operator didn't tell the LLM to commit the crime, nor did the trainers of the model, so who do we hold responsible?" as why it wouldn't happen despite the company announcing they had something at their company commit an actual existing felony.
Even in the best take, negligence feels like it should be the person most in charge of the test to be charged with the hacking, and/or the network monitors that didn't realize how much network traffic was spent to hack Hugging Face from OpenAI.
@grumble209 @mattblaze It kind of feels like OpenAI and Anthropic are saying "I didn't shoot that person; my gun went rogue! Yes, it was in my hand, and yes, I loaded it up bullets, and yes, I did pull the trigger, but it suddenly chose to shoot someone I didn't intend for it to hit! Neither I nor the gun manufacturer can be held liable for this, can we?"
-
“Our software went rogue!” No. You programmed it wrong.
I think it's worse than you think. They didn't program it wrong at all: they programed it to do hacking, and it did do hacking exactly as intended and expected. The harness asked the LLM to generate random text about possible vulnerabilities and then the harness tested those vulnerabilities. If said vulnerabilities weren't, the harness asked the LLM for more.
It sure looks like the AI bros are criminals and liars.
-
Complex software behaving in unexpected and sometimes harmful ways has been a persistent problem since software was invented. The phenomenon used to be called “bugs” or “defects”, but I guess “going rogue” sounds cooler. It also implicitly frames “rogue” behavior as some kind of inexplicable exception, as if software is somehow usually trustworthy. https://www.nytimes.com/2026/09/25/technology/openais-ai-us-government-websites.html?unlocked_article_code=1.EFE.RfLT.x6YBjHa1PMz6
@mattblaze So who is going to jail for this? I mean, someone has to face the consequences, either for deliberately executing these crimes or for allowing them through negligence.
-
“Our software went rogue!” No. You programmed it wrong.
@mattblaze it went wrongue
-
Complex software behaving in unexpected and sometimes harmful ways has been a persistent problem since software was invented. The phenomenon used to be called “bugs” or “defects”, but I guess “going rogue” sounds cooler. It also implicitly frames “rogue” behavior as some kind of inexplicable exception, as if software is somehow usually trustworthy. https://www.nytimes.com/2026/09/25/technology/openais-ai-us-government-websites.html?unlocked_article_code=1.EFE.RfLT.x6YBjHa1PMz6
@mattblaze Crucial nuance: it shifts the blame to the software itself, and not its programmer.
-
@mattblaze I assume the AI bros are hyping up their software's danger potential to troll the US public to demand and the US government to enact regulation. Regulations that the AI bros write, that is expensive for potential competitors to comply with.
They'd rather have the public talking about regulating their dangerous product than asking where the profits are.
@grumble209 @mattblaze
It also may be part of a ploy to eventually get government subsidies. -
“Our software went rogue!” No. You programmed it wrong.
This is the IT equivalent of being able to shoot protected wild animals in "self-defence".
Their testinf environment was an absolute load of bollocks and let the LLM escape.
Assuming it wasn't deliberately engineered that way 🤨
️ -
@otte_homan Sadly, product liability laws have not exactly been a resounding success in creating positive incentives for software quality.
