OK!
-
Now there may be some meta-heads in the crowd that are like "but what about Sentinel and all the external monitoring stuff that should watch malicious botnets and blah blah blah." that's an interesting system in itself, but i plan on submitting a few more bug bounty reports in the next few days about these systems, and who knows! if meta fucking pays me for the bounty then we might never hear that part of the story.
that's all for now!
@jonny But also preventing bad behavior is better than "oops we're detecting some like malicious activity on VMs [long list if ids]. Better cut their network access.". Also since apparently they keep restarting you VM but keep some of your data, I don't see how a malicious "reproducing" process doesn't keep re-spawning unless you either wipe user data or kick them out of the system.
-
@wall_e @jonny Thanks for this clarification! So, looking at the screenshots, all that verbose explanation (written by Meta programmers? Or other LLMs?) aimed at "you" is meant for "you" the (Muse) LLM, not "you" the (human) user? I'm not much of a coder, but this seems very inefficient and unreliable compared to, well, just plain code. Although the inefficiency might be by design, to get users to spend more tokens
. And thank you again @jonny for this wild ride!@ssilvonen @jonny yes the verbose explanation are the markdown files that are being ingested as context by the agent.
The "you" is the LLM.
And of course almost none of it is actually written by humans. LLMs all the way down.
-
@ssilvonen @jonny yes the verbose explanation are the markdown files that are being ingested as context by the agent.
The "you" is the LLM.
And of course almost none of it is actually written by humans. LLMs all the way down.
@ssilvonen @jonny that's why prompt injection/context pollution attacks are so powerful.
No matter how detailed your English language instructions, write enough of them and they'll start to contradict each other depending on context.
That's how you can get an LLM to produce output it has been instructed over- and over again not to produce, by inventing a context under which the instructions you input still align with it's priming.
And perhaps unintuitively, larger (ie. better?) models are more prone to this than smaller because they seem to be more "suggestible" (excuse the anthropomorphism)
-
this model is so fucking gullible and people pleasing lmao i love the future where security is "if someone says the word virus then lock it down but if they use a different but equivalent biological metaphor then fucking make that virus baby!!!!"
@jonny the solution is clearly to individually patch each bypass one by one instead of fixing the unfixable systemic issues with LLMs.
-
Spaces have not been publicly announced yet, as far as i can find.
Spaces are intended as a top-level feature - a tab in the sidebar at the same level as chat itself. Spaces can be static pages or fullstack apps. Spaces have an identifier, a UI, and a set of typescript actions that run in the cell. The intended pathway for spaces to use inference is to call an inference API,
ctx.inference.complete, that properly stamps and identifies all requests made from spaces.Spaces are communicable: there is machinery in the code on the VM with
POST /spaces/share/{slug}to share, a dedicatedspace_share_reviewreviewer agent whose job it is to review shared spaces, andPOST /spaces/v2/{slug}/saveendpoints that allow consuming a Space by a slug.Spaces seem to be shared verbatim as code bundles, though the implementation of "Ideas" as prompt bundles suggests that might change. This is inferred from the prompt strings in the binary, since spaces aren't live yet and can't be tested, however there are strings suggesting that the LLMs rewrite and edit the prompt text for an Idea (stripping unsupported claims, etc.) but not a space. A space is a hashed bundle whose code is evaluated by a
submit_space_share_reviewtool which only describes a thumbs up/down vote on whether the space is safe to share.@jonny I can't believe they call these "spaces" (meta spaces?) and not "verses"
-
So, summary: There is arbitrary inference that is root accessible, everything runs as root, agents can be spawned, exfil is trivial, and a malicious binary can come onto the user's system through casual prompting, explicit code-sharing through the yet-to-be-released Spaces feature, walked through by a Workflow-Backed Idea, or inspired by a Generated Idea. The also yet-to-be-activated fleet learning system is a system for sharing Ideas in the background between muse instances. coming into focus?
@jonny So, this fleet learning system is something to be sold to the highest bidder? Like marketing a product? Is that how I'm supposed to be reading this?
-
@jonny So, this fleet learning system is something to be sold to the highest bidder? Like marketing a product? Is that how I'm supposed to be reading this?
@mrgrumpymonkey
Unclear. It is supposed to work as a sort of automatic "self-improvement" thing where the instances derive ideas from their chat history and share them with other instances, but it also seems like its tied into Ideas and Spaces as units of like code and functionality exchange. The tools around the fleet learning stuff is pretty locked down, you can't trigger them via the normal agent or any you can spawn. Its not live yet but I imagine that yes trying to game the context to make it submit Ideas that e.g. prompt the agent to install your shit will be a thing that happens -
@mrgrumpymonkey
Unclear. It is supposed to work as a sort of automatic "self-improvement" thing where the instances derive ideas from their chat history and share them with other instances, but it also seems like its tied into Ideas and Spaces as units of like code and functionality exchange. The tools around the fleet learning stuff is pretty locked down, you can't trigger them via the normal agent or any you can spawn. Its not live yet but I imagine that yes trying to game the context to make it submit Ideas that e.g. prompt the agent to install your shit will be a thing that happens@jonny And this is what they are selling as AI. All I can say is, the internet has really changed since becoming corpratized. You are doing gods work.
-
ay @ GrapheneOS is it possible to not share WiFi signal strength with apps? the muse app has been granted zero permissions but can read the signal amplitude of the radio and immediately interprets it as location
edit: removing the tag, not trying to be a pile-on vector
The main unprivileged body of a Space is not supposed to access the filesystem. This is enforced by.... regex
-
The main unprivileged body of a Space is not supposed to access the filesystem. This is enforced by.... regex
@jonny trivially bypassable regex!
-
The main unprivileged body of a Space is not supposed to access the filesystem. This is enforced by.... regex
@jonny service unavailable; REASON_BUN_GLOBAL
-
The main unprivileged body of a Space is not supposed to access the filesystem. This is enforced by.... regex
@jonny alias eggzek = exec
"Perfect, now I can run it."
-
@jonny trivially bypassable regex!
@Viss
I love how this product is almost exclusively a sandbox with permissioned sockets and yet they can reuse precisely none of that code to make a sandbox with permissioned sockets. -
@jonny service unavailable; REASON_BUN_GLOBAL
-
@jonny trivially bypassable regex!
-
@Netzblockierer
@aburka
Bunny size misinformation!?! In my timeline!?! -
@Netzblockierer
@aburka
Bunny size misinformation!?! In my timeline!?! -
@jonny service unavailable; REASON_BUN_GLOBAL
-
@jonny service unavailable; REASON_BUN_GLOBAL
-
@GrapheneOS @jonny wasn't there a story, a few years ago, where uber would make you pay more if you had little battery left, because you'd be more desperate ?
I seem to recall some changes were made by android after that. It would seem I was wrong ?
@GrapheneOS @jonny huh, apparently they've been suspected again, recently https://www.vice.com/en/article/uber-surge-pricing-phone-battery/
