this story about openai "breaking containment" is such bullshit if you know anything about how computer security is supposed to be done.
-
if your "containment" is "telling the robot to be a good boy" you're not a serious person
first we got "prompt injection" aka "deciding to forget two decades of knowledge about secure composition of code and inputs, langsec, etc" and now we have "deciding to forget everything about capability-based security and writing a call of duty script about it"
-
first we got "prompt injection" aka "deciding to forget two decades of knowledge about secure composition of code and inputs, langsec, etc" and now we have "deciding to forget everything about capability-based security and writing a call of duty script about it"
"the robot went to a website" wow should we throw a party should we invite tim berners lee
-
this story about openai "breaking containment" is such bullshit if you know anything about how computer security is supposed to be done. if a program accesses a remote system it's because you decided to let it do that. LLMs don't have magical powers that can work around "we unplugged the ethernet"
@jcoglan it's a sign theyre running out of money when they gotta dial up the science fiction
-
if your "containment" is "telling the robot to be a good boy" you're not a serious person
@jcoglan i guess this is exactly what they did for “containment”
-
this story about openai "breaking containment" is such bullshit if you know anything about how computer security is supposed to be done. if a program accesses a remote system it's because you decided to let it do that. LLMs don't have magical powers that can work around "we unplugged the ethernet"
@jcoglan Running in a chroot or container or VM is no substitute for airgapping the computer.
-
if your "containment" is "telling the robot to be a good boy" you're not a serious person
@jcoglan "We just ask our uncleared staff nicely not to access the files that require security clearances"
-
this story about openai "breaking containment" is such bullshit if you know anything about how computer security is supposed to be done. if a program accesses a remote system it's because you decided to let it do that. LLMs don't have magical powers that can work around "we unplugged the ethernet"
@jcoglan Kind of. It wasn't totally isolated from the Internet but had a "a curated allowlist that permits routine package installation". The AI got past the allowlist, and then successfully attacked an external company. It's still potentially worrying, even if only because more people are likely to be running poorly-isolated hacking AIs. https://simonwillison.net/2026/Jul/22/openai-cyberattack/
-
@jcoglan Kind of. It wasn't totally isolated from the Internet but had a "a curated allowlist that permits routine package installation". The AI got past the allowlist, and then successfully attacked an external company. It's still potentially worrying, even if only because more people are likely to be running poorly-isolated hacking AIs. https://simonwillison.net/2026/Jul/22/openai-cyberattack/
@TheoEsc ok but that means they found a bug in the firewall, not that the AI "disobeyed its instructions" which is the framing of these types of stories, and one which makes no sense at a conceptual level
-
J jwcph@helvede.net shared this topic
-
@jcoglan Kind of. It wasn't totally isolated from the Internet but had a "a curated allowlist that permits routine package installation". The AI got past the allowlist, and then successfully attacked an external company. It's still potentially worrying, even if only because more people are likely to be running poorly-isolated hacking AIs. https://simonwillison.net/2026/Jul/22/openai-cyberattack/
"got passed the allowlist" still means it had access to arbitrarily execute code. And arbitrary code execution leads to... Arbitrary code execution.
-
this story about openai "breaking containment" is such bullshit if you know anything about how computer security is supposed to be done. if a program accesses a remote system it's because you decided to let it do that. LLMs don't have magical powers that can work around "we unplugged the ethernet"
@jcoglan yes it was bullshit - just hype for their latest fund raise, which was announced two days later (today):
https://londondaily.com/openai-secures-40-billion-funding-reaches-300-billion-valuation
#AI is such a pump n dump. -
this story about openai "breaking containment" is such bullshit if you know anything about how computer security is supposed to be done. if a program accesses a remote system it's because you decided to let it do that. LLMs don't have magical powers that can work around "we unplugged the ethernet"
@jcoglan Still no. They didn't "let it do that."
Someone, somewhere, a human, *told it to do that.*
-
R reynir@social.data.coop shared this topic
