I am convinced we are on the verge of the first "AI agent worm".
-
@csepp Don't forget about KeePassXC. I dunno if they kept going after this "initial test" or not https://www.reddit.com/r/KeePass/comments/1lnvw6q/keepassxc_codebases_jump_into_generative_ai/
@csepp And don't forget about LITERALLY MOZILLA FIREFOX
-
@aeva But once that was done, the agent was set up to install on users' devices
So the initial attack vector can literally be "Any AI agent in your stack whatsoever getting tricked" as a pathway for infecting computers everywhere
@cwebber apropos of nothing, is pottery still a big deal for humans? i was thinking this morning that pottery might be a nice career change for me.
-
@mcc exactly put
@cwebber @mcc @dandylyons
not forgetting the second post - the one that appropriately begins by "meanwhile" - wasn't conflating anything, it was contrasting the gravity of the situation with the surreallistically ingenuous state of mind of some people. -
@csepp And don't forget about LITERALLY MOZILLA FIREFOX
@cwebber Oh shit, I rely on all three of these.
Welppppp. I guess I'll have to start looking into alternative password managers. -
I am convinced we are on the verge of the first "AI agent worm". This looks like the closest hint of it, though it isn't it quite itself: an attack on a PR agent that got it to set up to install openclaw with full access on 4k machines https://grith.ai/blog/clinejection-when-your-ai-tool-installs-another
But, the agents installed weren't given instructions to *do* anything yet.
Soon they will be. And when they are, the havoc will be massive. Unlike traditional worms, where you're looking for the typically byte-for-byte identical worm embedded in the system, an agent worm can do different, nondeterministic things on every install, and carry out a global action.
I suspect we're months away from seeing the first agent worm, *if* that. There may already be some happening right now in FOSS projects, undetected.
Ah, the infinite papirclips scenario.
-
@cwebber Oh shit, I rely on all three of these.
Welppppp. I guess I'll have to start looking into alternative password managers. -
-
@mttaggart @mcc @cwebber Do we know what is being used for inference? At this point in time it's unlikely that they can use a self-hosted model, so there will be network calls.
-
-
@Canageek @csepp There was a recent thing, I can't find it now, where Mozilla added a commit to their agents thing to say "don't explicitly say when AI agents helped author a commit anymore", probably because they were getting community pushback
as you may have guessed, it got some community pushback
-
@cwebber apropos of nothing, is pottery still a big deal for humans? i was thinking this morning that pottery might be a nice career change for me.
-
@Canageek @csepp There was a recent thing, I can't find it now, where Mozilla added a commit to their agents thing to say "don't explicitly say when AI agents helped author a commit anymore", probably because they were getting community pushback
as you may have guessed, it got some community pushback
-
@mttaggart @mcc @cwebber Do we know what is being used for inference? At this point in time it's unlikely that they can use a self-hosted model, so there will be network calls.
@dvshkn @mcc @cwebber So the trick here is if you install OpenClaw in secret on a user's machine who isn't checking carefully, you might hide easily in network traffic. Use of tools like Claude Code would make the same API calls, which is likely for users who would be targeted with these attacks.
The real insane part is if multiple instance of OpenClaw were running on the same machine, so not even the process name looked suspicious. But of course process names are a poor indicator and can be changed.
-
-
@Canageek @csepp There was a recent thing, I can't find it now, where Mozilla added a commit to their agents thing to say "don't explicitly say when AI agents helped author a commit anymore", probably because they were getting community pushback
as you may have guessed, it got some community pushback
-
@mttaggart @mcc @cwebber Do we know what is being used for inference? At this point in time it's unlikely that they can use a self-hosted model, so there will be network calls.
@dvshkn @mttaggart @cwebber one thing i wonder is if it's in principle possible to firewall claude/copilot endpoints. in the old days of the internet this would have been possible, in the present day the claude/copilot api servers are probably mixed in with the aws/azure IP pool and more than likely move around…
-
@cwebber @csepp Yeah, I know Vivaldi has taken an anti-AI stance, but they're based on Chrome
AND from what I understand Servo Is nowhere near ready for end users, and based on every tech project I've ever liked will probably turn out to be either garbage or run by people who eat kittens or something by the time it comes out
-
@dvshkn @mttaggart @cwebber one thing i wonder is if it's in principle possible to firewall claude/copilot endpoints. in the old days of the internet this would have been possible, in the present day the claude/copilot api servers are probably mixed in with the aws/azure IP pool and more than likely move around…
-
I am convinced we are on the verge of the first "AI agent worm". This looks like the closest hint of it, though it isn't it quite itself: an attack on a PR agent that got it to set up to install openclaw with full access on 4k machines https://grith.ai/blog/clinejection-when-your-ai-tool-installs-another
But, the agents installed weren't given instructions to *do* anything yet.
Soon they will be. And when they are, the havoc will be massive. Unlike traditional worms, where you're looking for the typically byte-for-byte identical worm embedded in the system, an agent worm can do different, nondeterministic things on every install, and carry out a global action.
I suspect we're months away from seeing the first agent worm, *if* that. There may already be some happening right now in FOSS projects, undetected.
@cwebber TBF, I hope for a huge thing. If it's small, nothing will change in now people run those programs and there will be a million smaller hacks instead.
-
@mttaggart @dvshkn @cwebber …that… should have occurred to me. I guess I got too used to the threat model of "is Windows 10 phoning home / searching bing without telling me", where Microsoft has the ability to ship IP lists. Probably only Microsoft can really do this.
… I guess if the attacker really thought ahead they could do DNS lookup through the firefox DoH server or something but they don't have much reason to try that.
