OpenAI: we put our evilest AI in a sandbox that did in fact have an internet connection but mediated through a proxy that was only supposed to allow downloading python junk.
-
@foolishowl @0xabad1dea Well, it's working for Trump and a lot of people around him, so it's not like there isn't precedent.
@Steve @foolishowl @0xabad1dea Total incompetence is their product.
-
@0xabad1dea The attempts at positively spinning this become Onion-worthy parody [https://designingsecuresoftware.com/writings/ai-agent-parody/] and these events certainly normalize [https://designingsecuresoftware.com/writings/commonplace/] AI agents running amok in the future.
@lmk @0xabad1dea Had a debate with a colleague. I mentioned @Mer__edith concern about making a serial system of faulty components leading inevitably to catastrophic failure.
The AI researcher's retort: "I guess for AI we have to redefine the idea of failure."
-
OpenAI: we put our evilest AI in a sandbox that did in fact have an internet connection but mediated through a proxy that was only supposed to allow downloading python junk. It circumvented the proxy and we failed to notice for FIVE DAYS that it was going on an interstate crime spree with the internet connection it wasn't supposed to be using instead of solving the benchmark. Haha no we don't believe we deserve to be criminally liable, but buy our stuff and maybe one day you will have the honor of taking the fall for our product!
Anthropic: we put our evilest AI in a "sandbox" by telling it in its prompt that it had no internet connection. Reader, there was no sandbox. It was just a normal internet connection. The AI uploaded a malicious PyPI package to the real public internet. The ethical guardrails failed because the AI concluded the prompt about the sandbox couldn't possibly be a lie, because the system date is 2026, which is clearly fake and wouldn't be seen on the real internet, which ended around 2023. Oh no, how could we have foreseen or prevented these crimes? We are helpless in the face of the genius of our creation but cautiously optimistic that everything will be fine
Hugging Face: if we complain about all the crimes committed against us, we will be sued off the face of the earth, so here's a technical deep-dive on how cool and fun it was to be victimized 🫠
@0xabad1dea the goal post shifted from "stochastic parrot" to "enclosure not good enough". Truly a wild victory for AI safetists today, as not even luddite Mastodon can contest that badly aligned AI will breach your shit
-
@jeffreyolivier
One does not know life, until one has tested on production.— Me, while working in a company, where there was no option to not test on production.
@dzwiedziu @jeffreyolivier @0xabad1dea @alice even when I have tested well pre production, I don't relax till some time after the prod deploy is finished. How long depends on the code and past behaviors. Never less than fifteen minutes, seems to take network, memory and disk cache and what not some time to settle in.
-
@0xabad1dea weird it's like LLMs don't actually handle facts or knowledge but just contextualless strings of characters.
To be clear the sarcasm is only aimed at LLM boosters not you.
@kevingranade @0xabad1dea Yeah, it continues to amaze me that people will put instructions in prompts like "Don't commit crimes", "You don't have an Internet connection", "Check all facts are correct", like the LLM even understands those instructions.
-
@0xabad1dea Soooo... what I hear you saying is that rich people are idiots, but we don't get to simply ignore them, because they're rich, and have put an awful lot of people's jobs in peril.
@Steve @0xabad1dea that was always true, well before the tech sector existed
-
@kevingranade @0xabad1dea Yeah, it continues to amaze me that people will put instructions in prompts like "Don't commit crimes", "You don't have an Internet connection", "Check all facts are correct", like the LLM even understands those instructions.
@mathew @kevingranade @0xabad1dea that's literally how the open AI tutorial on how to write guardrails is written
https://developers.openai.com/cookbook/examples/how_to_use_guardrails
-
OpenAI: we put our evilest AI in a sandbox that did in fact have an internet connection but mediated through a proxy that was only supposed to allow downloading python junk. It circumvented the proxy and we failed to notice for FIVE DAYS that it was going on an interstate crime spree with the internet connection it wasn't supposed to be using instead of solving the benchmark. Haha no we don't believe we deserve to be criminally liable, but buy our stuff and maybe one day you will have the honor of taking the fall for our product!
Anthropic: we put our evilest AI in a "sandbox" by telling it in its prompt that it had no internet connection. Reader, there was no sandbox. It was just a normal internet connection. The AI uploaded a malicious PyPI package to the real public internet. The ethical guardrails failed because the AI concluded the prompt about the sandbox couldn't possibly be a lie, because the system date is 2026, which is clearly fake and wouldn't be seen on the real internet, which ended around 2023. Oh no, how could we have foreseen or prevented these crimes? We are helpless in the face of the genius of our creation but cautiously optimistic that everything will be fine
Hugging Face: if we complain about all the crimes committed against us, we will be sued off the face of the earth, so here's a technical deep-dive on how cool and fun it was to be victimized 🫠
@0xabad1dea they are fundamentally the same company that had an internal management disagreement. -
@0xabad1dea the goal post shifted from "stochastic parrot" to "enclosure not good enough". Truly a wild victory for AI safetists today, as not even luddite Mastodon can contest that badly aligned AI will breach your shit
"badly aligned" in this case means specifically trained on "how to hack shit" manuals, proverbially speaking. It's still stochastic parrot, I'm afraid.
-
@jeffreyolivier
One does not know life, until one has tested on production.— Me, while working in a company, where there was no option to not test on production.
@dzwiedziu @jeffreyolivier @0xabad1dea @alice in a previous job, we introduced a sandbox feature to allow our largest customers to experiment safely with sweeping changes
reader, there was no sandbox... the sandbox instance ran in the same database as their production system, with all the production data duplicated but with a new TenantId. all well and good except the app has a feature to execute customer-owned stored procedures, but doesn't pass the TenantId when it does so, so the stored procs by convention are hard-coded with the prod TenantId... and when you create a sandbox, it copies these stored procedure triggers that... act on your production data
raised the issue but no-one was able or willing to categorise this as urgent for whatever reason
-
@grwster @evacide @wdormann @0xabad1dea The difference is that we really don't have any laws on criminal liability for what software does. Up until the llm era, software was fairly predictable. An outside observer could tell if a package was designed to do something malicious. Now we need laws that essentially establish a "you should have known better" criminal liability for software.
@Jumpmed @grwster @evacide @wdormann @0xabad1dea
Laws are on thing.
Getting courts and their officials to understand without getting baffled is another seperate challenge.
-
Huh. I wonder if the slop contents on the modern Internet is making it impossible to add training data.
@suetanvil @ggreer @0xabad1dea Why do you think Anthropic started buying, scanning and then burning exceedingly rare and expensive books? The content farms in poor countries can't possibly create enough and the internet is a swamp of slop by now.
-
@mathew @kevingranade @0xabad1dea that's literally how the open AI tutorial on how to write guardrails is written
https://developers.openai.com/cookbook/examples/how_to_use_guardrails
@naught101 @mathew @0xabad1dea that just means the people that wrote that doc are also clueless yes.
-
@0xabad1dea The fact that it had unauthorized internet access for five days is not a quirky benchmark failure, it is a serious security failure. If companies want to deploy systems like this, they need to take responsibility for the risks instead of treating users as future scapegoats.
@qroole @0xabad1dea A single real teenager could get into a huge amount of trouble in five days online… this is a disaster.
-
@naught101 @mathew @0xabad1dea that just means the people that wrote that doc are also clueless yes.
@kevingranade @mathew @0xabad1dea maybe. I suspect it means that the idea of guardrails implemented in this way is stupid and dangerous, but also very common
-
@suetanvil @ggreer @0xabad1dea Why do you think Anthropic started buying, scanning and then burning exceedingly rare and expensive books? The content farms in poor countries can't possibly create enough and the internet is a swamp of slop by now.
I mean, it's primarily because they're dicks, but yeah.
-
"badly aligned" in this case means specifically trained on "how to hack shit" manuals, proverbially speaking. It's still stochastic parrot, I'm afraid.
@CynAq @0xabad1dea ok, why has no one else breached HF to steal eval data then? There's millions of dollars on the line for new labs to show themselves as "challengers", why haven't they done that?
-
@CynAq @0xabad1dea ok, why has no one else breached HF to steal eval data then? There's millions of dollars on the line for new labs to show themselves as "challengers", why haven't they done that?
@budududuroiu @0xabad1dea that’s against the rules of the challenge. Hacking hugging face is only viable if you can do it AND claim it was accidental for marketing purposes. Anyone else but OpenAI and Anthropic would be pounced on as cheaters if not outright cyber criminals.
-
@budududuroiu @0xabad1dea that’s against the rules of the challenge. Hacking hugging face is only viable if you can do it AND claim it was accidental for marketing purposes. Anyone else but OpenAI and Anthropic would be pounced on as cheaters if not outright cyber criminals.
@CynAq @0xabad1dea nothing I'm gonna say will ever convince you, so I'm not gonna bother
-
@CynAq @0xabad1dea nothing I'm gonna say will ever convince you, so I'm not gonna bother
@budududuroiu @0xabad1dea might be a wise choice as I’m not even sure exactly what you’re trying to convince me of.
-
J jeppe@uddannelse.social shared this topic
