No, OpenAI's new magic models did not autonomously hack Huggingface.
-
@michelin well they worked together with Huggingface (on PR and telling them about a bug they may or may not have found). So unless Huggingface sues them who would attack OpenAI?
-
@Gurre @tante No to defend anyone but more to take that concern from you (I would also hate if they start using Alien universe names). The company is named after the hugging face emoji https://en.wikipedia.org/wiki/Hugging_Face
Their Public Relations department can say that all they want.
But we know what it really stands for.
-
Their Public Relations department can say that all they want.
But we know what it really stands for.
Open AI's statement,
"These deployment safeguards were intentionally not enabled during this evaluation because it was aimed at testing cyber vulnerabilities."
makes me think of all the actions of all the "corporate types" in the Alien franchise too.
And that's *never* a good thing.
-
@tante How Tagesschau is reporting about it can only be described as refusal to practice journalism. 7 out of 9 paragraphs say "Open AI says" and the other two quote nameless "Experts" and Anthropic's product. And tomorrow I have to discuss with someone again about the possibility of consciousness in token generator software https://www.tagesschau.de/wirtschaft/unternehmen/openai-ki-hackerangriff-100.html
@miles_leif @tante this is how 99% of the press is dealing with AI since the beginning of the hype, a hype they contribute to feed.
-
No, OpenAI's new magic models did not autonomously hack Huggingface.
Per OpenAI's PR blog post (https://openai.com/index/hugging-face-model-evaluation-security-incident/
"This incident occurred during an internal evaluation which prompts models to pursue advanced exploitation using complex attack paths, in an effort to quantify their cyber capabilities. We estimate maximal cyber capabilities by running this evaluation without production classifiers used to prevent models from pursuing high-risk cyber activity."
So they prompted their model running without guardrails to hack some shit. OpenAI told it to do that, the model didn't do shit autonomously (no LLM ever does anything autonomously, it's always prompted).
The whole story is PR. We know that from Anthopic's Mythos: "Look we have this super secret new model and it's so powerful. We are scared ourselves. And you will soon be able to rent it!"
-
No, OpenAI's new magic models did not autonomously hack Huggingface.
Per OpenAI's PR blog post (https://openai.com/index/hugging-face-model-evaluation-security-incident/
"This incident occurred during an internal evaluation which prompts models to pursue advanced exploitation using complex attack paths, in an effort to quantify their cyber capabilities. We estimate maximal cyber capabilities by running this evaluation without production classifiers used to prevent models from pursuing high-risk cyber activity."
So they prompted their model running without guardrails to hack some shit. OpenAI told it to do that, the model didn't do shit autonomously (no LLM ever does anything autonomously, it's always prompted).
The whole story is PR. We know that from Anthopic's Mythos: "Look we have this super secret new model and it's so powerful. We are scared ourselves. And you will soon be able to rent it!"
yesterday NPR had a long story on the new hot battery powered pickup truck from the startup
https://www.slate.auto/enand the NPR piece was full of the most hilarious nonsense, all provided to NPR by Slate's excellent PR team
shrug
-
No, OpenAI's new magic models did not autonomously hack Huggingface.
Per OpenAI's PR blog post (https://openai.com/index/hugging-face-model-evaluation-security-incident/
"This incident occurred during an internal evaluation which prompts models to pursue advanced exploitation using complex attack paths, in an effort to quantify their cyber capabilities. We estimate maximal cyber capabilities by running this evaluation without production classifiers used to prevent models from pursuing high-risk cyber activity."
So they prompted their model running without guardrails to hack some shit. OpenAI told it to do that, the model didn't do shit autonomously (no LLM ever does anything autonomously, it's always prompted).
The whole story is PR. We know that from Anthopic's Mythos: "Look we have this super secret new model and it's so powerful. We are scared ourselves. And you will soon be able to rent it!"
@tante "OpenAI told it to do that, the model didn't do shit autonomously [...]"
Exactly, what one should have expected (and how it turned out before every time). -
@tante How Tagesschau is reporting about it can only be described as refusal to practice journalism. 7 out of 9 paragraphs say "Open AI says" and the other two quote nameless "Experts" and Anthropic's product. And tomorrow I have to discuss with someone again about the possibility of consciousness in token generator software https://www.tagesschau.de/wirtschaft/unternehmen/openai-ki-hackerangriff-100.html
@miles_leif @tante Not the only one unfortunately, sigh
-
@tante a friend suggests it's co-marketing with Huggingface, if you look at the timeline of announcements
Yeh, they're both chatbot companies.
-
'How did it get the "stolen credentials"?'
@JeffGrigg @SonOfSunTzu @hagen @tante
password.txt -
@SonOfSunTzu @tante that’s what mythos did as well though? you point at something and say go and if you’re willing to pay for compute it goes. it literally did what it was meant to do – and was explicitly told to do. what’s the news here?
@hagen @tante assuming a genuine question ... AIUI Mythos carried out tests in a controlled and authorised way, directly answering the tests it was set.
OpenAI's "collection of models" hacked out of a sandbox, and from there hacked into a different organisation, when given a set of tests it was meant to solve inside the sandbox.
Kind of as if a student sitting an exam broke into the exam board's office to steal the answers - so it's different in terms of alignment, and legal responsibility...
-
-
No, OpenAI's new magic models did not autonomously hack Huggingface.
Per OpenAI's PR blog post (https://openai.com/index/hugging-face-model-evaluation-security-incident/
"This incident occurred during an internal evaluation which prompts models to pursue advanced exploitation using complex attack paths, in an effort to quantify their cyber capabilities. We estimate maximal cyber capabilities by running this evaluation without production classifiers used to prevent models from pursuing high-risk cyber activity."
So they prompted their model running without guardrails to hack some shit. OpenAI told it to do that, the model didn't do shit autonomously (no LLM ever does anything autonomously, it's always prompted).
The whole story is PR. We know that from Anthopic's Mythos: "Look we have this super secret new model and it's so powerful. We are scared ourselves. And you will soon be able to rent it!"
"no LLM ever does anything autonomously, it's always prompted)."
Subgoals. You give it a complex task and it tries to solve it by breaking it down in subgoals... That it invents all itself...
Sure this COULD be a PR stunt. But then a lot of employees from 2 different companies would have to be in on the conspiracy... Not so likely
-
"no LLM ever does anything autonomously, it's always prompted)."
Subgoals. You give it a complex task and it tries to solve it by breaking it down in subgoals... That it invents all itself...
Sure this COULD be a PR stunt. But then a lot of employees from 2 different companies would have to be in on the conspiracy... Not so likely
-
No, OpenAI's new magic models did not autonomously hack Huggingface.
Per OpenAI's PR blog post (https://openai.com/index/hugging-face-model-evaluation-security-incident/
"This incident occurred during an internal evaluation which prompts models to pursue advanced exploitation using complex attack paths, in an effort to quantify their cyber capabilities. We estimate maximal cyber capabilities by running this evaluation without production classifiers used to prevent models from pursuing high-risk cyber activity."
So they prompted their model running without guardrails to hack some shit. OpenAI told it to do that, the model didn't do shit autonomously (no LLM ever does anything autonomously, it's always prompted).
The whole story is PR. We know that from Anthopic's Mythos: "Look we have this super secret new model and it's so powerful. We are scared ourselves. And you will soon be able to rent it!"
@tante Anything to keep the billions flowing. What a disgrace.
-
No, OpenAI's new magic models did not autonomously hack Huggingface.
Per OpenAI's PR blog post (https://openai.com/index/hugging-face-model-evaluation-security-incident/
"This incident occurred during an internal evaluation which prompts models to pursue advanced exploitation using complex attack paths, in an effort to quantify their cyber capabilities. We estimate maximal cyber capabilities by running this evaluation without production classifiers used to prevent models from pursuing high-risk cyber activity."
So they prompted their model running without guardrails to hack some shit. OpenAI told it to do that, the model didn't do shit autonomously (no LLM ever does anything autonomously, it's always prompted).
The whole story is PR. We know that from Anthopic's Mythos: "Look we have this super secret new model and it's so powerful. We are scared ourselves. And you will soon be able to rent it!"
@tante Indeed. As I put it yesterday, OpenAI’s agent “broke containment” the same way my cuckoo clock will break containment if I put it in front of a button.
-
To me it rather seems that the AI-denial is a cult..
Bergman:
https://youtu.be/KpTZbq-eV38?is=yIA9reuuiFsgwaRO
And with the "humiliations argument" we now have an explanation for it:
-
To me it rather seems that the AI-denial is a cult..
Bergman:
https://youtu.be/KpTZbq-eV38?is=yIA9reuuiFsgwaRO
And with the "humiliations argument" we now have an explanation for it:
-
To me it rather seems that the AI-denial is a cult..
Bergman:
https://youtu.be/KpTZbq-eV38?is=yIA9reuuiFsgwaRO
And with the "humiliations argument" we now have an explanation for it:
-
No, OpenAI's new magic models did not autonomously hack Huggingface.
Per OpenAI's PR blog post (https://openai.com/index/hugging-face-model-evaluation-security-incident/
"This incident occurred during an internal evaluation which prompts models to pursue advanced exploitation using complex attack paths, in an effort to quantify their cyber capabilities. We estimate maximal cyber capabilities by running this evaluation without production classifiers used to prevent models from pursuing high-risk cyber activity."
So they prompted their model running without guardrails to hack some shit. OpenAI told it to do that, the model didn't do shit autonomously (no LLM ever does anything autonomously, it's always prompted).
The whole story is PR. We know that from Anthopic's Mythos: "Look we have this super secret new model and it's so powerful. We are scared ourselves. And you will soon be able to rent it!"
@tante
- i do not think it's unlikely that the prompt said "get a good score on this benchmark" and the model wildly misinterpreted the goal, as anyone who uses llms sees them do all the time
- even if the prompt was "hack huggingface", the fact that it did that without further prompting is noteworthy.
