I am convinced we are on the verge of the first "AI agent worm".
-
@bituur_esztreym @lispi314 @cwebber it's a reference https://www.youtube.com/watch?v=F9OmTnuLzeQ
-
@cwebber so I'm following this right, it sounds like the project or its maintainers don't even necessarily need to even be using LLM tools, the attack pattern simply targets contributors who are using LLM development tools? and so all that is really needed is for the payload to be subtle and the maintainer to be sufficiently overwhelmed (say, by an endless fire hose of LLM-generated liquid shit slop pull requests)?
-
@bituur_esztreym @lispi314 @cwebber it's a reference https://www.youtube.com/watch?v=F9OmTnuLzeQ
-
@cwebber apropos of nothing, is pottery still a big deal for humans? i was thinking this morning that pottery might be a nice career change for me.
-
@aeva@mastodon.gamedev.place @cwebber@social.coop Not really, it’s been mass-industrialized so at this point outside of Etsy stuff you can largely forget it.
And no one’s going to use very expensive handmade pottery, it’s going to be a display piece.
-
-
-
@aeva I'm a huge fan of ceramics, in general, as an art form. Side effect of being the neurodivergent son of an archaeologist.
If you start making pieces, please give me a chirp! -
I am convinced we are on the verge of the first "AI agent worm". This looks like the closest hint of it, though it isn't it quite itself: an attack on a PR agent that got it to set up to install openclaw with full access on 4k machines https://grith.ai/blog/clinejection-when-your-ai-tool-installs-another
But, the agents installed weren't given instructions to *do* anything yet.
Soon they will be. And when they are, the havoc will be massive. Unlike traditional worms, where you're looking for the typically byte-for-byte identical worm embedded in the system, an agent worm can do different, nondeterministic things on every install, and carry out a global action.
I suspect we're months away from seeing the first agent worm, *if* that. There may already be some happening right now in FOSS projects, undetected.
-
I am convinced we are on the verge of the first "AI agent worm". This looks like the closest hint of it, though it isn't it quite itself: an attack on a PR agent that got it to set up to install openclaw with full access on 4k machines https://grith.ai/blog/clinejection-when-your-ai-tool-installs-another
But, the agents installed weren't given instructions to *do* anything yet.
Soon they will be. And when they are, the havoc will be massive. Unlike traditional worms, where you're looking for the typically byte-for-byte identical worm embedded in the system, an agent worm can do different, nondeterministic things on every install, and carry out a global action.
I suspect we're months away from seeing the first agent worm, *if* that. There may already be some happening right now in FOSS projects, undetected.
@cwebber
As foretold by SF author John Brunner in _The Shockwave Rider_ (1975). -
@aeva I'm a huge fan of ceramics, in general, as an art form. Side effect of being the neurodivergent son of an archaeologist.
If you start making pieces, please give me a chirp!@aaron I'm probably going to pick up some supplies and experiment. I doubt I'll be selling anything anytime soon, but I'll post progress photos. I haven't worked with ceramics since '09 but I was alright at it back then.
-
@aaron I'm probably going to pick up some supplies and experiment. I doubt I'll be selling anything anytime soon, but I'll post progress photos. I haven't worked with ceramics since '09 but I was alright at it back then.
@aaron this is the only ceramic piece i still have from back then. not necessarily representative of what I'm likely to make now, but i like it
-
@aaron this is the only ceramic piece i still have from back then. not necessarily representative of what I'm likely to make now, but i like it
@aaron all of my sculpture since then has been 3D printed stuff. you can see an assortment in this old intro post https://mastodon.gamedev.place/@aeva/109307134622156004
-
I know some people are thinking "well pulling off this kind of thing, it would have to be controlled with intent of a human actor"
It doesn't have to be.
1. A human could *kick off* such a process, and then it runs away from them.
2. It wouldn't even require a specific prompt to kick off a worm. There's enough scifi out there for this to be something any one of the barely-monitored openclaw agents could determine it should do.Whether it's kicked off by a human explicitly or a stray agent, it doesn't require "intentionality". Biological viruses don't have interiority / intentionality, and yet are major threats that reproduce and adapt.
@cwebber the only bit from the post I slightly disagree with is that "Wrapping agents in sandboxes is tough to do", or rather that this should be left as a conclusion for people instead of emphasising as a prerequisite before using an agent (even if done imperfectly). Well, even to do development using package managers nowadays I guess...
My attempt at this: https://www.danieldemmel.me/blog/coding-agents-in-secured-vscode-dev-containers
