I'm posting here what I had written to a news outlet in April during the whole Mythos hype 🙄 and people keep on being like BUT why are you not making CLEAR arguments against Bla BLa Bla?
-
I'm posting here what I had written to a news outlet in April during the whole Mythos hype
and people keep on being like BUT why are you not making CLEAR arguments against Bla BLa Bla? Cause I'm learning that its a trap to make me waste my time debunking claims while they get trillions to implement their dystopian future and we're stuck talking about whatever they're doing rather than implementing our visions. 🧵 -
I'm posting here what I had written to a news outlet in April during the whole Mythos hype
and people keep on being like BUT why are you not making CLEAR arguments against Bla BLa Bla? Cause I'm learning that its a trap to make me waste my time debunking claims while they get trillions to implement their dystopian future and we're stuck talking about whatever they're doing rather than implementing our visions. 🧵My concerns were on the types of claims made about Mythos. There has been a growing trend of companies making claims and these claims being repeated without any way to verify them or having misleading ways of evaluating their claims. Each time we have had the actual code base, data and models to analyze, we have found a lot of issues with claims in many different domains of machine learning. My own work has shown this in other scenarios.
-
My concerns were on the types of claims made about Mythos. There has been a growing trend of companies making claims and these claims being repeated without any way to verify them or having misleading ways of evaluating their claims. Each time we have had the actual code base, data and models to analyze, we have found a lot of issues with claims in many different domains of machine learning. My own work has shown this in other scenarios.
For Mythos, for example, Anthropic doesn't tell us the number of false positives their tool returns, i.e. the number of times their tool says that something is a vulnerability and it ends up not being. My security expert collaborators tell me that this is one of the most important metrics by which security tools are judged, because it tells you the difference between a useful tool and a useless one that engineers won't use.
-
For Mythos, for example, Anthropic doesn't tell us the number of false positives their tool returns, i.e. the number of times their tool says that something is a vulnerability and it ends up not being. My security expert collaborators tell me that this is one of the most important metrics by which security tools are judged, because it tells you the difference between a useful tool and a useless one that engineers won't use.
Anthropic also claims that Mythos can replace security experts. It's one thing to claim that you've built a useful tool, another to claim that you can replace experts. Some security experts have even said that it's dishonest to say your tool is superior to security experts because it found bugs in old codebases and we don't know how often people audit them for bugs and fix them. Again a misleading claim that is repeated by those outside of the company.
-
Anthropic also claims that Mythos can replace security experts. It's one thing to claim that you've built a useful tool, another to claim that you can replace experts. Some security experts have even said that it's dishonest to say your tool is superior to security experts because it found bugs in old codebases and we don't know how often people audit them for bugs and fix them. Again a misleading claim that is repeated by those outside of the company.
Another thing that I find funny is that Anthropic is making all these claims about security while they themselves couldn't stop their own source code from leaking. And from analyzing that code, people have seen how things are done now with "vibe coding" using Claude. Even things that can be done simply are done with brute force, i.e. trying all possible scenarios before arriving at an answer or solution.
-
Another thing that I find funny is that Anthropic is making all these claims about security while they themselves couldn't stop their own source code from leaking. And from analyzing that code, people have seen how things are done now with "vibe coding" using Claude. Even things that can be done simply are done with brute force, i.e. trying all possible scenarios before arriving at an answer or solution.
How much computational power does it take to find any of these vulnerabilities Anthropic says they found? How much money would someone have to pay to use their tool vs bonafide experts?
-
How much computational power does it take to find any of these vulnerabilities Anthropic says they found? How much money would someone have to pay to use their tool vs bonafide experts?
A lot of security and safety is about processes, checks, clear people in charge of clear things, and clear access limits. That goes out the window with these “agents. It becomes harder to identify where things went wrong, or what types of tests you need to do to check for vulnerabilities and who is in charge of which issue.
-
A lot of security and safety is about processes, checks, clear people in charge of clear things, and clear access limits. That goes out the window with these “agents. It becomes harder to identify where things went wrong, or what types of tests you need to do to check for vulnerabilities and who is in charge of which issue.
Some experienced software engineers talked about how it's easy to just press “accept” of the buggy code that you see generated by Claude. So the question is, what about the new vulnerabilities created by using these “agents”? We’ve seen so many examples of issues, like people wiping out their entire production data.
-
Some experienced software engineers talked about how it's easy to just press “accept” of the buggy code that you see generated by Claude. So the question is, what about the new vulnerabilities created by using these “agents”? We’ve seen so many examples of issues, like people wiping out their entire production data.
Finally, I'm wondering why Anthropic is boasting about the security capabilities of Mythos and leaning into the cold war narrative (talking about Russia and China e.g.) Heidi Klaff (again security expert) told me that this has one them back favor from the US government after they were on the bad side with the whole pentagon thing. Anthropic hasn't told us whether mythos is better at this task than prior models. Why this and why now?
-
Some experienced software engineers talked about how it's easy to just press “accept” of the buggy code that you see generated by Claude. So the question is, what about the new vulnerabilities created by using these “agents”? We’ve seen so many examples of issues, like people wiping out their entire production data.
@timnitGebru The fact they would use it in the first place means I have questions.
Sincerely, an experienced software engineer...
-
I'm posting here what I had written to a news outlet in April during the whole Mythos hype
and people keep on being like BUT why are you not making CLEAR arguments against Bla BLa Bla? Cause I'm learning that its a trap to make me waste my time debunking claims while they get trillions to implement their dystopian future and we're stuck talking about whatever they're doing rather than implementing our visions. 🧵@timnitGebru just here to validate that it can most definitely be a trap because that's another bad faith tactic.
-
Some experienced software engineers talked about how it's easy to just press “accept” of the buggy code that you see generated by Claude. So the question is, what about the new vulnerabilities created by using these “agents”? We’ve seen so many examples of issues, like people wiping out their entire production data.
@timnitGebru ai bros want to freak out about watermarking but won't put that same energy towards the concept that ai companies could wilfully introduce backdoors in the same manner
-
Some experienced software engineers talked about how it's easy to just press “accept” of the buggy code that you see generated by Claude. So the question is, what about the new vulnerabilities created by using these “agents”? We’ve seen so many examples of issues, like people wiping out their entire production data.
@timnitGebru @Em0nM4stodon They can’t hear you, they’ve got their fingers in their ears
-
F folfdk@helvede.net shared this topic
