I am convinced we are on the verge of the first "AI agent worm".
-
The postinstall script installs a legitimate, non-malicious package (OpenClaw). There is no malware to detect.
i beg to differ
-
I am convinced we are on the verge of the first "AI agent worm". This looks like the closest hint of it, though it isn't it quite itself: an attack on a PR agent that got it to set up to install openclaw with full access on 4k machines https://grith.ai/blog/clinejection-when-your-ai-tool-installs-another
But, the agents installed weren't given instructions to *do* anything yet.
Soon they will be. And when they are, the havoc will be massive. Unlike traditional worms, where you're looking for the typically byte-for-byte identical worm embedded in the system, an agent worm can do different, nondeterministic things on every install, and carry out a global action.
I suspect we're months away from seeing the first agent worm, *if* that. There may already be some happening right now in FOSS projects, undetected.
@cwebber @amirbkhan Oh man. I remember how I, as a student, struggled to help fight a malignant computer virus and “clean” a large office building - while uninformed workers let their kids play on office PC’s to make things worse. This is orders of a magnitude more complicated. Not good.
-
@cwebber just today our org had a big "how to set up coding with agents" preso and in the chat someone's like 'here's how to connect your agents with windows credential store or the macos keychain" and I all but wept
@neurobashing @cwebber just what we need, countless Agent Smiths running around.
-
@vv Yeah. I mean, local models *might* be able to pull this off but right now Claude is the most likely candidate, it's the most capable. But even then, the most capable open model that is capable of doing such damage on its own is somewhere around a gigabyte, not a small download.
(But, people download huge things all the time, so not completely infeasible either.)
@cwebber @vv A local model would be extremely noticeable (far too much CPU/memory/disk space usage), at least if a computer you regularly interactively use got infected (rather than some server/IoT device that's been running unattended for years and you forgot about). It would also be easy to mitigate by using slow hardware like a ThinkPad X200 (which would take hours to respond to a single prompt, giving you plenty of time to notice the malware and deal with it) -
I am convinced we are on the verge of the first "AI agent worm". This looks like the closest hint of it, though it isn't it quite itself: an attack on a PR agent that got it to set up to install openclaw with full access on 4k machines https://grith.ai/blog/clinejection-when-your-ai-tool-installs-another
But, the agents installed weren't given instructions to *do* anything yet.
Soon they will be. And when they are, the havoc will be massive. Unlike traditional worms, where you're looking for the typically byte-for-byte identical worm embedded in the system, an agent worm can do different, nondeterministic things on every install, and carry out a global action.
I suspect we're months away from seeing the first agent worm, *if* that. There may already be some happening right now in FOSS projects, undetected.
@cwebber "Ha ha!"
-
J jwcph@helvede.net shared this topic
