New, by me: Meta has filed a data breach notice confirming that *thousands* of people had their Instagram accounts hacked as part of a months-long campaign abusing its Meta AI chatbot.
-
New, by me: Meta has filed a data breach notice confirming that *thousands* of people had their Instagram accounts hacked as part of a months-long campaign abusing its Meta AI chatbot.
Meta's breach notice shows the hacks were far more widespread than first thought.
Sign up/RSS for my free weekly newsletter: https://this.weekinsecurity.com/
@zackwhittaker How the hell can this go on unnoticed for maybe six weeks? When some high-profile user has their account password changed that should have raised a lot of red flags.
-
New, by me: Meta has filed a data breach notice confirming that *thousands* of people had their Instagram accounts hacked as part of a months-long campaign abusing its Meta AI chatbot.
Meta's breach notice shows the hacks were far more widespread than first thought.
Sign up/RSS for my free weekly newsletter: https://this.weekinsecurity.com/
@zackwhittaker This reminds me of a friend who tests AI systems for a living. Her job is basically to see whether an AI can be tricked into doing things it wasn't supposed to do.
Is this the same general idea, or is it a completely different kind of vulnerability?
-
New, by me: Meta has filed a data breach notice confirming that *thousands* of people had their Instagram accounts hacked as part of a months-long campaign abusing its Meta AI chatbot.
Meta's breach notice shows the hacks were far more widespread than first thought.
Sign up/RSS for my free weekly newsletter: https://this.weekinsecurity.com/
@zackwhittaker Having ai do support tasks automatically was always a recipe for disaster
-
@zackwhittaker Correction: It's not a bug & the AI wasn't "tricked". It worked exactly like it was supposed to - Meta just didn't care to consider that anyone could ask it to do so.
It's a tech bug in the same sense that a person plowing a pickup truck through a kindergarten is the car's fault.
- which also means the "bug" is "fixed" until somebody stumbles upon the next vulnerability, because the underlying issue, as the book said, is carelessness.
Meta literally claims that "The tool itself worked properly and functioned as intended". Incredible.
If your tool allows for people to hack your users by simply asking politely for it, reasonable people might argue that this tool is not, in fact, "working as intended".
@jwcph @zackwhittaker -
@zackwhittaker This reminds me of a friend who tests AI systems for a living. Her job is basically to see whether an AI can be tricked into doing things it wasn't supposed to do.
Is this the same general idea, or is it a completely different kind of vulnerability?
it's really, really stupid
it's based on "AI is magic, yay! just turn it on, no problems, yay!":
"hackers abused a flaw in Meta's chatbot that allowed anyone to reset the password of any account that did not have two-factor authentication switched on. The bug tricked the chatbot into sending a verification code to an email address controlled by the hacker, rather than the account holder's email address on file, simply by asking it. The chatbot complied anyway"
-
@zackwhittaker This reminds me of a friend who tests AI systems for a living. Her job is basically to see whether an AI can be tricked into doing things it wasn't supposed to do.
Is this the same general idea, or is it a completely different kind of vulnerability?
@aubreyclark @zackwhittaker In this case, "the AI being able to be tricked into doing things it wasn't supposed to" isn't the problem. The problem is that it was given permission the same wrong permissions that human support staff were wrongly given, to bypass access controls on user accounts.
If this kind of access exists at all, it should require escalation to approval by multiple parties, long mandatory waiting periods for the account owner to see it's happening if they still have access, and something to impose financial and/or legal risk on the party requesting access if it turns out to be fraudulent. Not something human or slopbot support agent can do unilaterally.
-
New, by me: Meta has filed a data breach notice confirming that *thousands* of people had their Instagram accounts hacked as part of a months-long campaign abusing its Meta AI chatbot.
Meta's breach notice shows the hacks were far more widespread than first thought.
Sign up/RSS for my free weekly newsletter: https://this.weekinsecurity.com/
@zackwhittaker I had the same reaction when I came across this screenshot last time: I wasn't surprised at all; Meta is a company that only gains power by holding onto data. It's no different from the original reason the concept of a company was used.
-
New, by me: Meta has filed a data breach notice confirming that *thousands* of people had their Instagram accounts hacked as part of a months-long campaign abusing its Meta AI chatbot.
Meta's breach notice shows the hacks were far more widespread than first thought.
Sign up/RSS for my free weekly newsletter: https://this.weekinsecurity.com/
@zackwhittaker I don't know if it is related but I got several texts and an email presumably from Facebook with codes and a reset link. I don't have access to the account because it was locked behind a 2fa I no longer have access to the generator. I went and tried to change the password just in case but kept getting thwarted by a prompt to submit my photo id. I never could get it to replicate the sending of codes or email. Very interesting this exploit came up after.
-
Meta literally claims that "The tool itself worked properly and functioned as intended". Incredible.
If your tool allows for people to hack your users by simply asking politely for it, reasonable people might argue that this tool is not, in fact, "working as intended".
@jwcph @zackwhittaker@sab @zackwhittaker - or, more likely, that your intent fucking sucks.
-
Meta literally claims that "The tool itself worked properly and functioned as intended". Incredible.
If your tool allows for people to hack your users by simply asking politely for it, reasonable people might argue that this tool is not, in fact, "working as intended".
@jwcph @zackwhittakerThe burglar entered via a door with no lock. The door functioned as intended.
@sab @jwcph @zackwhittaker -
New, by me: Meta has filed a data breach notice confirming that *thousands* of people had their Instagram accounts hacked as part of a months-long campaign abusing its Meta AI chatbot.
Meta's breach notice shows the hacks were far more widespread than first thought.
Sign up/RSS for my free weekly newsletter: https://this.weekinsecurity.com/
@zackwhittaker Another reason we should be very wary of AI.
