<![CDATA[Complex software behaving in unexpected and sometimes harmful ways has been a persistent problem since software was invented.]]>Complex software behaving in unexpected and sometimes harmful ways has been a persistent problem since software was invented. The phenomenon used to be called “bugs” or “defects”, but I guess “going rogue” sounds cooler. It also implicitly frames “rogue” behavior as some kind of inexplicable exception, as if software is somehow usually trustworthy. https://www.nytimes.com/2026/09/25/technology/openais-ai-us-government-websites.html?unlocked_article_code=1.EFE.RfLT.x6YBjHa1PMz6

]]>https://forum.fedi.dk/topic/f7bb4241-1717-4816-ac1b-8d17894edf87/complex-software-behaving-in-unexpected-and-sometimes-harmful-ways-has-been-a-persistent-problem-since-software-was-invented.RSS for NodeMon, 28 Sep 2026 14:13:20 GMTSat, 26 Sep 2026 07:25:33 GMT60<![CDATA[Reply to Complex software behaving in unexpected and sometimes harmful ways has been a persistent problem since software was invented. on Sat, 26 Sep 2026 11:38:24 GMT]]>@mattblaze Shut them down. They are computer viruses.

]]>https://forum.fedi.dk/post/https://social.vivaldi.net/users/gbsills/statuses/117337142337217317https://forum.fedi.dk/post/https://social.vivaldi.net/users/gbsills/statuses/117337142337217317Sat, 26 Sep 2026 11:38:24 GMT<![CDATA[Reply to Complex software behaving in unexpected and sometimes harmful ways has been a persistent problem since software was invented. on Sat, 26 Sep 2026 11:27:48 GMT]]>@mattblaze Doesn't saying it was going "rogue" make it sounds more threatening like it has some form of thought process or an independent task to fulfil separate from the one assigned to it? It has none of those things of course, it was sent out to do the thing and did it but it's much more scary if it seems uncontrollable doesn't it?

]]>https://forum.fedi.dk/post/https://oldbytes.space/users/nini/statuses/117337100694815725https://forum.fedi.dk/post/https://oldbytes.space/users/nini/statuses/117337100694815725Sat, 26 Sep 2026 11:27:48 GMT<![CDATA[Reply to Complex software behaving in unexpected and sometimes harmful ways has been a persistent problem since software was invented. on Sat, 26 Sep 2026 11:10:40 GMT]]>@mattblaze Why do I get the feeling that @simonzerafa is more right than he thinks, that this was deliberate? The question becomes why... is this just stochastic hooliganism, or is this an attempt to bring a failed ponzi scheme to a close by blaming it on the inevitable regulations?

]]>https://forum.fedi.dk/post/https://hachyderm.io/users/stonebear2/statuses/117337033334774999https://forum.fedi.dk/post/https://hachyderm.io/users/stonebear2/statuses/117337033334774999Sat, 26 Sep 2026 11:10:40 GMT<![CDATA[Reply to Complex software behaving in unexpected and sometimes harmful ways has been a persistent problem since software was invented. on Sat, 26 Sep 2026 10:56:53 GMT]]>@mattblaze at some point a programmer put in an option. “ if you don’t have valid credentials find valid ones” it was programmed to cheat, by someone. When VW programmed engines to know when the were being tested and decrease exhaust nobody said the engines went rouge.

]]>https://forum.fedi.dk/post/https://mastodon.social/users/passwordsarehard4/statuses/117336979116943920https://forum.fedi.dk/post/https://mastodon.social/users/passwordsarehard4/statuses/117336979116943920Sat, 26 Sep 2026 10:56:53 GMT<![CDATA[Reply to Complex software behaving in unexpected and sometimes harmful ways has been a persistent problem since software was invented. on Sat, 26 Sep 2026 09:56:39 GMT]]>@mattblaze
"...as if software is somehow  usually trustworthy."
You're probably familiar with the UK Post Office Horizon scandal where some people assumed software is trustworthy. It didn't end well for them (and unfortunately for a lot of innocent people as well).

]]>https://forum.fedi.dk/post/https://mastodon.nz/users/ThomasPonnet/statuses/117336742257269764https://forum.fedi.dk/post/https://mastodon.nz/users/ThomasPonnet/statuses/117336742257269764Sat, 26 Sep 2026 09:56:39 GMT<![CDATA[Reply to Complex software behaving in unexpected and sometimes harmful ways has been a persistent problem since software was invented. on Sat, 26 Sep 2026 09:36:30 GMT]]>@mattblaze

So a computer crime was committed.
There is a suspect.

Usually the police comes in and confiscates ALL computers that might have been used or possibly mightz be used for a complete crime. Like copying a disney song.

Similarly the police won't just raid and secure only the gun used in a murder, but all firfearms in possession of the suspect.

So why not at OpenAI, Anthropic or Google?
Then the threat will be eliminate, too.
Win-win!

]]>https://forum.fedi.dk/post/https://chaos.social/users/vampirdaddy/statuses/117336663056806216https://forum.fedi.dk/post/https://chaos.social/users/vampirdaddy/statuses/117336663056806216Sat, 26 Sep 2026 09:36:30 GMT<![CDATA[Reply to Complex software behaving in unexpected and sometimes harmful ways has been a persistent problem since software was invented. on Sat, 26 Sep 2026 09:24:37 GMT]]>@mattblaze it also suggests agency on the part of the software (look how smart it is!)- also as if the authors didn’t have anything to do with it

]]>https://forum.fedi.dk/post/https://mastodon.social/users/fodwyer/statuses/117336616303010001https://forum.fedi.dk/post/https://mastodon.social/users/fodwyer/statuses/117336616303010001Sat, 26 Sep 2026 09:24:37 GMT<![CDATA[Reply to Complex software behaving in unexpected and sometimes harmful ways has been a persistent problem since software was invented. on Sat, 26 Sep 2026 09:19:55 GMT]]>@otte_homan Sadly, product liability laws have not exactly been a resounding success in creating positive incentives for software quality.

]]>https://forum.fedi.dk/post/https://federate.social/users/mattblaze/statuses/117336597839511277https://forum.fedi.dk/post/https://federate.social/users/mattblaze/statuses/117336597839511277Sat, 26 Sep 2026 09:19:55 GMT<![CDATA[Reply to Complex software behaving in unexpected and sometimes harmful ways has been a persistent problem since software was invented. on Sat, 26 Sep 2026 08:56:52 GMT]]>@mattblaze

This is the IT equivalent of being able to shoot protected wild animals in "self-defence".

Their testinf environment was an absolute load of bollocks and let the LLM escape.

Assuming it wasn't deliberately engineered that way 🤨🤷‍♂️

]]>https://forum.fedi.dk/post/https://infosec.exchange/users/simonzerafa/statuses/117336507179194361https://forum.fedi.dk/post/https://infosec.exchange/users/simonzerafa/statuses/117336507179194361Sat, 26 Sep 2026 08:56:52 GMT<![CDATA[Reply to Complex software behaving in unexpected and sometimes harmful ways has been a persistent problem since software was invented. on Sat, 26 Sep 2026 08:55:38 GMT]]>@grumble209 @mattblaze
It also may be part of a ploy to eventually get government subsidies.

]]>https://forum.fedi.dk/post/https://mastodon.social/users/brouhaha/statuses/117336502310051824https://forum.fedi.dk/post/https://mastodon.social/users/brouhaha/statuses/117336502310051824Sat, 26 Sep 2026 08:55:38 GMT<![CDATA[Reply to Complex software behaving in unexpected and sometimes harmful ways has been a persistent problem since software was invented. on Sat, 26 Sep 2026 08:55:04 GMT]]>@mattblaze Crucial nuance: it shifts the blame to the software itself, and not its programmer.

]]>https://forum.fedi.dk/post/https://infosec.exchange/users/karl/statuses/117336500140033096https://forum.fedi.dk/post/https://infosec.exchange/users/karl/statuses/117336500140033096Sat, 26 Sep 2026 08:55:04 GMT<![CDATA[Reply to Complex software behaving in unexpected and sometimes harmful ways has been a persistent problem since software was invented. on Sat, 26 Sep 2026 08:35:50 GMT]]>@mattblaze it went wrongue

]]>https://forum.fedi.dk/post/https://fux.am/users/amlor/statuses/01M3EDPEYQS48CQ0AWMH527FAJhttps://forum.fedi.dk/post/https://fux.am/users/amlor/statuses/01M3EDPEYQS48CQ0AWMH527FAJSat, 26 Sep 2026 08:35:50 GMT<![CDATA[Reply to Complex software behaving in unexpected and sometimes harmful ways has been a persistent problem since software was invented. on Sat, 26 Sep 2026 08:29:33 GMT]]>@mattblaze So who is going to jail for this? I mean, someone has to face the consequences, either for deliberately executing these crimes or for allowing them through negligence.

]]>https://forum.fedi.dk/post/https://social.cologne/ap/users/116232768628584081/statuses/117336399747806061https://forum.fedi.dk/post/https://social.cologne/ap/users/116232768628584081/statuses/117336399747806061Sat, 26 Sep 2026 08:29:33 GMT<![CDATA[Reply to Complex software behaving in unexpected and sometimes harmful ways has been a persistent problem since software was invented. on Sat, 26 Sep 2026 08:21:27 GMT]]>@mattblaze

I think it's worse than you think. They didn't program it wrong at all: they programed it to do hacking, and it did do hacking exactly as intended and expected. The harness asked the LLM to generate random text about possible vulnerabilities and then the harness tested those vulnerabilities. If said vulnerabilities weren't, the harness asked the LLM for more.

It sure looks like the AI bros are criminals and liars.

]]>https://forum.fedi.dk/post/https://mastodon.mit.edu/users/djl/statuses/117336367913758125https://forum.fedi.dk/post/https://mastodon.mit.edu/users/djl/statuses/117336367913758125Sat, 26 Sep 2026 08:21:27 GMT<![CDATA[Reply to Complex software behaving in unexpected and sometimes harmful ways has been a persistent problem since software was invented. on Sat, 26 Sep 2026 08:15:59 GMT]]>@grumble209 @mattblaze It kind of feels like OpenAI and Anthropic are saying "I didn't shoot that person; my gun went rogue! Yes, it was in my hand, and yes, I loaded it up bullets, and yes, I did pull the trigger, but it suddenly chose to shoot someone I didn't intend for it to hit! Neither I nor the gun manufacturer can be held liable for this, can we?"

]]>https://forum.fedi.dk/post/https://mstdn.ca/users/AT1ST/statuses/117336346415145320https://forum.fedi.dk/post/https://mstdn.ca/users/AT1ST/statuses/117336346415145320Sat, 26 Sep 2026 08:15:59 GMT<![CDATA[Reply to Complex software behaving in unexpected and sometimes harmful ways has been a persistent problem since software was invented. on Sat, 26 Sep 2026 08:11:47 GMT]]>@grumble209 @mattblaze Another angle on this is to try and avoid culpability for the crimes they let their LLM A.I. agents do. Like, the Hugging Face hack that OpenAI did had a lot of people thinking "Wait - isn't hacking already illegal? It's a felony, isn't it? Why wasn't someone at OpenAI charged with that crime?", a lot of people claimed "Well, the operator didn't tell the LLM to commit the crime, nor did the trainers of the model, so who do we hold responsible?" as why it wouldn't happen despite the company announcing they had something at their company commit an actual existing felony.

Even in the best take, negligence feels like it should be the person most in charge of the test to be charged with the hacking, and/or the network monitors that didn't realize how much network traffic was spent to hack Hugging Face from OpenAI.

]]>https://forum.fedi.dk/post/https://mstdn.ca/users/AT1ST/statuses/117336329901758879https://forum.fedi.dk/post/https://mstdn.ca/users/AT1ST/statuses/117336329901758879Sat, 26 Sep 2026 08:11:47 GMT<![CDATA[Reply to Complex software behaving in unexpected and sometimes harmful ways has been a persistent problem since software was invented. on Sat, 26 Sep 2026 08:04:53 GMT]]>@mattblaze
When is Judgement Day ? (Financial reports indicating the fraud).
Supposed to be in October.
All this shit talk just to avoid it.

]]>https://forum.fedi.dk/post/https://piaille.fr/users/OrangeR/statuses/117336302777230796https://forum.fedi.dk/post/https://piaille.fr/users/OrangeR/statuses/117336302777230796Sat, 26 Sep 2026 08:04:53 GMT<![CDATA[Reply to Complex software behaving in unexpected and sometimes harmful ways has been a persistent problem since software was invented. on Sat, 26 Sep 2026 08:03:29 GMT]]>@mattblaze
" It's not a bug, it's a feature! " .

]]>https://forum.fedi.dk/post/https://mstdn.social/users/JorisMeys/statuses/117336297266001754https://forum.fedi.dk/post/https://mstdn.social/users/JorisMeys/statuses/117336297266001754Sat, 26 Sep 2026 08:03:29 GMT<![CDATA[Reply to Complex software behaving in unexpected and sometimes harmful ways has been a persistent problem since software was invented. on Sat, 26 Sep 2026 08:03:01 GMT]]>@quinn @mattblaze

The best side effect of the AI is now it's possible to blame it for almost every human error.

«Sorry, officer, not my fault, but the AI pushed the car at 84 mph!»

]]>https://forum.fedi.dk/post/https://mastodon.cisti.org/users/GustavinoBevilacqua/statuses/117336295440931722https://forum.fedi.dk/post/https://mastodon.cisti.org/users/GustavinoBevilacqua/statuses/117336295440931722Sat, 26 Sep 2026 08:03:01 GMT<![CDATA[Reply to Complex software behaving in unexpected and sometimes harmful ways has been a persistent problem since software was invented. on Sat, 26 Sep 2026 08:01:27 GMT]]>@quinn @mattblaze

plausible denAIability

]]>https://forum.fedi.dk/post/https://toot.si/users/hook/statuses/117336289266012152https://forum.fedi.dk/post/https://toot.si/users/hook/statuses/117336289266012152Sat, 26 Sep 2026 08:01:27 GMT<![CDATA[Reply to Complex software behaving in unexpected and sometimes harmful ways has been a persistent problem since software was invented. on Sat, 26 Sep 2026 07:57:36 GMT]]>@mattblaze

]]>https://forum.fedi.dk/post/https://infosec.exchange/users/aristot73/statuses/117336274117430401https://forum.fedi.dk/post/https://infosec.exchange/users/aristot73/statuses/117336274117430401Sat, 26 Sep 2026 07:57:36 GMT<![CDATA[Reply to Complex software behaving in unexpected and sometimes harmful ways has been a persistent problem since software was invented. on Sat, 26 Sep 2026 07:54:29 GMT]]>@mattblaze it wasn't programmed wrong, it's kind of designed to do this. It's either: they are brazenly drumming up issues to boost stock prices or it's a consequence of firing/not hiring in the first place people who know how to manage these situations (so they can boost stock prices).

Or it's just an attempt to boost stock prices. It might also be to do with stock prices.

]]>https://forum.fedi.dk/post/https://mastodonapp.uk/users/flipper/statuses/117336261868707228https://forum.fedi.dk/post/https://mastodonapp.uk/users/flipper/statuses/117336261868707228Sat, 26 Sep 2026 07:54:29 GMT<![CDATA[Reply to Complex software behaving in unexpected and sometimes harmful ways has been a persistent problem since software was invented. on Sat, 26 Sep 2026 07:52:25 GMT]]>@mattblaze rude, stop attacking my plausible deniability

]]>https://forum.fedi.dk/post/https://social.circl.lu/users/quinn/statuses/117336253733493756https://forum.fedi.dk/post/https://social.circl.lu/users/quinn/statuses/117336253733493756Sat, 26 Sep 2026 07:52:25 GMT<![CDATA[Reply to Complex software behaving in unexpected and sometimes harmful ways has been a persistent problem since software was invented. on Sat, 26 Sep 2026 07:40:30 GMT]]>@grumble209 I’m not sure exactly what their game is here, but large incumbents LOVE gatekeeping regulation.

]]>https://forum.fedi.dk/post/https://federate.social/users/mattblaze/statuses/117336206884645444https://forum.fedi.dk/post/https://federate.social/users/mattblaze/statuses/117336206884645444Sat, 26 Sep 2026 07:40:30 GMT