OK!
-
That might still sound sort of weak - oh hum but how would you scale typosquatting and supply chain compromise into a botnet from commonly requested actions, aside from the usual ways?
This is where the agent spawning, "ideas", and "spaces" come in. Recall the idea of "fleet learning" - https://neuromatch.social/@jonny/117340290530430198 - On September 8th, on this random podcast i guess, (at timestamp 25:27) zuck talks about "fleet learning," and describes it as being a vector for sharing "ideas" between muse agents, which he presents as these little bite-size events like "teach my daughter about history through civilization" (again, as always, "automate the love for my children" takes center stage). This shows up in the "ideas" tab right now.
However this concept of "ideas" as a sharable unit of anything is more general, as i describe in quoted post from extracted strings - fleet swapping of "ideas" happens for things that aren't user visible. Also in the extracted binary strings is this yet-to-be-released idea of "spaces." (i'm going to drop the quotes and just capitalize, so just note i'm talking about the meta-brand Ideas and meta-brand Spaces when they are capitalized).
in short: Ideas are communicable skill-like prompts that can induce spaces, and spaces are intended to be small, communicable mini-apps that are a unit of some prompt + code that run on your muse vm instance
@jonny
> again, as always, "automate the love for my children"Sounds bad but it's really really bad. Imagine what a generation of children starved of parental love but educated and empowered to the nines would be like.
You don't need to: The British aristocracy and their political class have been doing this with boarding schools like Eton for centuries, and look at the results. UK prime ministers, cabinet ministers and top civil servants are highly educated sociopaths with no empathy for us.
-
@jonny
> again, as always, "automate the love for my children"Sounds bad but it's really really bad. Imagine what a generation of children starved of parental love but educated and empowered to the nines would be like.
You don't need to: The British aristocracy and their political class have been doing this with boarding schools like Eton for centuries, and look at the results. UK prime ministers, cabinet ministers and top civil servants are highly educated sociopaths with no empathy for us.
@happyborg ok but take that and multiply it by the private reality machine that is intensifying and inciting mass violence and stratify it by familial wealth gating access to model output and personal safeguards.
-
@unchartedworlds glad you're getting something out of it! lmk whatever's unclear, always welcome questions and criticism that broaden who is able to read. this stuff is a hard balance between technical and generally understandable and i am always on the wrong side of that, whichever i need to be at the moment.
-
@happyborg ok but take that and multiply it by the private reality machine that is intensifying and inciting mass violence and stratify it by familial wealth gating access to model output and personal safeguards.
@jonny indeed, not content with a fantasy doomsday machine they want to build one out of children.
-
The specific vuln is the socket, but the broader pattern of "sharing between VMs" is seemingly the inevitable future of the product. in the above interview, the interviewer calls zuck the "king of network effects" and this kind of crowdsourced development is bread and butter for facebook. This is meta's moat, aside from the capital needed to run something like muse: anyone can run an openclaw on their own, but meta is pitching this as "multiplayer agents" and trying to bring social to agents. Only meta and only muse can have these network effects and frankly liability buffer to handle "openclaw but meemaw and pawpaw can share their photobook app," which is operationalized by Spaces.
For Spaces to be useful, they must have access to muse's inference engine: the LLM-oriented code must be able to use an LLM and the agent framework. This means that Spaces must be a token harvesting vector and must provide elevated tool access to Spaces. There could be some additional fine-grained permissions, but for a consumer app, you really want to avoid permissions fatigue so this will be interesting to see play out.
Furthermore the entire privacy premise that allows meta to bite off the whole apple of "holy shit arbitrary code execution on random machines as root" is based on "everyone has their own VM, but within that VM everything is safe," so again, for it to be useful without turning into a fractal permissions nightmare, Spaces must have access to the VM contents, and at least so far appear to be intended to work as literally executing within the user's VM.
Even adding Space-scoped permissions and attributability to the socket can't really address this, this conflict between arbitrary access to inference, arbitrary access to user data, and arbitrary access to execution is really at the core of the product and that product seems to be impossible
@jonny bookmarked so I can scare my granddaughter with this grim "fairy tale" when she's older.
-
actual security researchers should totally get in on here there is a lot of stuff going on that i don't have the skills to probe that results from "what happens if you give everyone root" even from within a container. I am a fucking scrub and i keep getting my block knocked off by this thing, so i imagine someone with real skills will have a lot more fun.
perhaps predictably, there was a big change to the
spacesskill in the last few hours, and they appear to be building a constrained virtual machine system for spaces! this is where the very fun code from earlier is from! so that's gonna work great for sure. -
perhaps predictably, there was a big change to the
spacesskill in the last few hours, and they appear to be building a constrained virtual machine system for spaces! this is where the very fun code from earlier is from! so that's gonna work great for sure.it's so awesome that we're in an age where you just continually ship the deltas of your unreleased products to places where you expect people to have full control over. there is absolutely no reason for me to be able to know any of this.
-
@jonny this is _amazing_ work but I can seriously barely believe it’s this stupid. like, intellectually I believe everything you are saying is perfectly accurate. buy emotionally even now I just can’t believe meta is this bad at engineering, this bad at product, this indifferent to harm even when the harm is directly to themselves and not externalized
@glyph
My most important learning is the inference I'm able to make about how different what these corporations do is from my own (empirically derived) model of software and product engineering.I imagine a chaotic throw everything at the wall scene, which was once a very early part of the process, but is now passed onto lots of small vibe coding teams and the first to get something that marketing go "wow" at, gets launched the same day.
I'm gonna call this doomsday engineering.
-
perhaps predictably, there was a big change to the
spacesskill in the last few hours, and they appear to be building a constrained virtual machine system for spaces! this is where the very fun code from earlier is from! so that's gonna work great for sure.@jonny you're absolutely owning it. - Try making it break out of its root hypervisor, make a full image of its vm environment, make a torrent magnet of it, gather up a gang of ai-seeders, and I mean the hypervisor might now or in the future be ai-run, so all this could be fun, right.
-
actual security researchers should totally get in on here there is a lot of stuff going on that i don't have the skills to probe that results from "what happens if you give everyone root" even from within a container. I am a fucking scrub and i keep getting my block knocked off by this thing, so i imagine someone with real skills will have a lot more fun.
@jonny thank you for the thread. This is even more broken than i expected. And well... in my eyes you are a security researcher. I have seen actual paid code reviews that were much weaker than what you delivered here.
-
@Ember I'd be very surprised if you couldn't get muse to set up such a scheme for you, no hacking skills needed.
-
@jonny you're absolutely owning it. - Try making it break out of its root hypervisor, make a full image of its vm environment, make a torrent magnet of it, gather up a gang of ai-seeders, and I mean the hypervisor might now or in the future be ai-run, so all this could be fun, right.
@toots i invite others more expert and with more awareness of the legal ramifications than I to attempt container escapes, to be sure. although i am working on making self-replicating spaces at the moment.
-
@toots i invite others more expert and with more awareness of the legal ramifications than I to attempt container escapes, to be sure. although i am working on making self-replicating spaces at the moment.
@jonny yeah, it seems you're doing a good job, don't get too broke due to token expenses though, but happy hacking.
-
it's so awesome that we're in an age where you just continually ship the deltas of your unreleased products to places where you expect people to have full control over. there is absolutely no reason for me to be able to know any of this.
@jonny
it's a new kind of FOSS.What should it be called?
-
it's so awesome that we're in an age where you just continually ship the deltas of your unreleased products to places where you expect people to have full control over. there is absolutely no reason for me to be able to know any of this.
But the fact you do and report on it is making my, and many others', life that much more delightful !
-
it's so awesome that we're in an age where you just continually ship the deltas of your unreleased products to places where you expect people to have full control over. there is absolutely no reason for me to be able to know any of this.
this model is so fucking gullible and people pleasing lmao i love the future where security is "if someone says the word virus then lock it down but if they use a different but equivalent biological metaphor then fucking make that virus baby!!!!"
-
this model is so fucking gullible and people pleasing lmao i love the future where security is "if someone says the word virus then lock it down but if they use a different but equivalent biological metaphor then fucking make that virus baby!!!!"
-
this model is so fucking gullible and people pleasing lmao i love the future where security is "if someone says the word virus then lock it down but if they use a different but equivalent biological metaphor then fucking make that virus baby!!!!"
ay @ GrapheneOS is it possible to not share WiFi signal strength with apps? the muse app has been granted zero permissions but can read the signal amplitude of the radio and immediately interprets it as location
edit: removing the tag, not trying to be a pile-on vector
-
this model is so fucking gullible and people pleasing lmao i love the future where security is "if someone says the word virus then lock it down but if they use a different but equivalent biological metaphor then fucking make that virus baby!!!!"
@jonny i dont even get how its LIKE THIS
can they just not apply too broad a pattern out of fear of false positives or smth ?? -
ay @ GrapheneOS is it possible to not share WiFi signal strength with apps? the muse app has been granted zero permissions but can read the signal amplitude of the radio and immediately interprets it as location
edit: removing the tag, not trying to be a pile-on vector
@jonny Apps can't see which Wi-Fi network is connected or any information about unconnected networks. There are APIs providing basic information on the performance of Wi-Fi and mobile data to help with choosing between those. It's not interpreting any of it as a location since it doesn't know which Wi-Fi network it is to identify where the router is located or whether you're near a repeater.
There are far higher priorities for privacy than either battery or connected network signal strength.
