OK!
-
this model is so fucking gullible and people pleasing lmao i love the future where security is "if someone says the word virus then lock it down but if they use a different but equivalent biological metaphor then fucking make that virus baby!!!!"
@jonny the solution is clearly to individually patch each bypass one by one instead of fixing the unfixable systemic issues with LLMs.
-
Spaces have not been publicly announced yet, as far as i can find.
Spaces are intended as a top-level feature - a tab in the sidebar at the same level as chat itself. Spaces can be static pages or fullstack apps. Spaces have an identifier, a UI, and a set of typescript actions that run in the cell. The intended pathway for spaces to use inference is to call an inference API,
ctx.inference.complete, that properly stamps and identifies all requests made from spaces.Spaces are communicable: there is machinery in the code on the VM with
POST /spaces/share/{slug}to share, a dedicatedspace_share_reviewreviewer agent whose job it is to review shared spaces, andPOST /spaces/v2/{slug}/saveendpoints that allow consuming a Space by a slug.Spaces seem to be shared verbatim as code bundles, though the implementation of "Ideas" as prompt bundles suggests that might change. This is inferred from the prompt strings in the binary, since spaces aren't live yet and can't be tested, however there are strings suggesting that the LLMs rewrite and edit the prompt text for an Idea (stripping unsupported claims, etc.) but not a space. A space is a hashed bundle whose code is evaluated by a
submit_space_share_reviewtool which only describes a thumbs up/down vote on whether the space is safe to share.@jonny I can't believe they call these "spaces" (meta spaces?) and not "verses"
-
So, summary: There is arbitrary inference that is root accessible, everything runs as root, agents can be spawned, exfil is trivial, and a malicious binary can come onto the user's system through casual prompting, explicit code-sharing through the yet-to-be-released Spaces feature, walked through by a Workflow-Backed Idea, or inspired by a Generated Idea. The also yet-to-be-activated fleet learning system is a system for sharing Ideas in the background between muse instances. coming into focus?
@jonny So, this fleet learning system is something to be sold to the highest bidder? Like marketing a product? Is that how I'm supposed to be reading this?
-
@jonny So, this fleet learning system is something to be sold to the highest bidder? Like marketing a product? Is that how I'm supposed to be reading this?
@mrgrumpymonkey
Unclear. It is supposed to work as a sort of automatic "self-improvement" thing where the instances derive ideas from their chat history and share them with other instances, but it also seems like its tied into Ideas and Spaces as units of like code and functionality exchange. The tools around the fleet learning stuff is pretty locked down, you can't trigger them via the normal agent or any you can spawn. Its not live yet but I imagine that yes trying to game the context to make it submit Ideas that e.g. prompt the agent to install your shit will be a thing that happens -
@mrgrumpymonkey
Unclear. It is supposed to work as a sort of automatic "self-improvement" thing where the instances derive ideas from their chat history and share them with other instances, but it also seems like its tied into Ideas and Spaces as units of like code and functionality exchange. The tools around the fleet learning stuff is pretty locked down, you can't trigger them via the normal agent or any you can spawn. Its not live yet but I imagine that yes trying to game the context to make it submit Ideas that e.g. prompt the agent to install your shit will be a thing that happens@jonny And this is what they are selling as AI. All I can say is, the internet has really changed since becoming corpratized. You are doing gods work.
-
ay @ GrapheneOS is it possible to not share WiFi signal strength with apps? the muse app has been granted zero permissions but can read the signal amplitude of the radio and immediately interprets it as location
edit: removing the tag, not trying to be a pile-on vector
The main unprivileged body of a Space is not supposed to access the filesystem. This is enforced by.... regex
-
The main unprivileged body of a Space is not supposed to access the filesystem. This is enforced by.... regex
@jonny trivially bypassable regex!
-
The main unprivileged body of a Space is not supposed to access the filesystem. This is enforced by.... regex
@jonny service unavailable; REASON_BUN_GLOBAL
-
The main unprivileged body of a Space is not supposed to access the filesystem. This is enforced by.... regex
@jonny alias eggzek = exec
"Perfect, now I can run it."
-
@jonny trivially bypassable regex!
@Viss
I love how this product is almost exclusively a sandbox with permissioned sockets and yet they can reuse precisely none of that code to make a sandbox with permissioned sockets. -
@jonny service unavailable; REASON_BUN_GLOBAL
-
@jonny trivially bypassable regex!
-
@Netzblockierer
@aburka
Bunny size misinformation!?! In my timeline!?! -
@Netzblockierer
@aburka
Bunny size misinformation!?! In my timeline!?! -
@jonny service unavailable; REASON_BUN_GLOBAL
-
@jonny service unavailable; REASON_BUN_GLOBAL
-
@GrapheneOS @jonny wasn't there a story, a few years ago, where uber would make you pay more if you had little battery left, because you'd be more desperate ?
I seem to recall some changes were made by android after that. It would seem I was wrong ?
@GrapheneOS @jonny huh, apparently they've been suspected again, recently https://www.vice.com/en/article/uber-surge-pricing-phone-battery/
-
@jonny The Location permission combined with the right low-level permission requests provides access to a lot of information on the nearby Wi-Fi networks. Without the location permission, there's only info on the signal strength of the best available currently connected cellular and WI-Fi networks.
It would definitely be possible for us to change this by offering having spoofed values. However, it would make no sense to work on this when far more important privacy issues exist.
@GrapheneOS Or you could argue that these are fundamentals. You can choose not to use apps, browsers, etc. You can’t choose not to use the battery or network.
-
The main unprivileged body of a Space is not supposed to access the filesystem. This is enforced by.... regex
@jonny the takeaway of your thread seems to be that regex can truly solve *any* problem. The duct tape of programming
️ -
RE: https://neuromatch.social/@jonny/117339825958098508
OK! Meta evaluated this as intended behavior, not applicable for a bug bounty, so therefore responsible disclosure no longer applies so here goes:
any process run within the VM can access the socket that provides inference with no attribution mechanism. This includes raw inference with arbitrary system and user prompts, as well as the ability to spawn agents with a toolset labeled as being for the "spaces" feature, which we will come back to.
This amounts to a horizontally contagious token and information harvesting bug being labeled as intended behavior.
Splitting details into new thread below
@jonny This whole thread is so life-affirming.
️
