OK!
-
@unchartedworlds glad you're getting something out of it! lmk whatever's unclear, always welcome questions and criticism that broaden who is able to read. this stuff is a hard balance between technical and generally understandable and i am always on the wrong side of that, whichever i need to be at the moment.
-
@happyborg ok but take that and multiply it by the private reality machine that is intensifying and inciting mass violence and stratify it by familial wealth gating access to model output and personal safeguards.
@jonny indeed, not content with a fantasy doomsday machine they want to build one out of children.
-
The specific vuln is the socket, but the broader pattern of "sharing between VMs" is seemingly the inevitable future of the product. in the above interview, the interviewer calls zuck the "king of network effects" and this kind of crowdsourced development is bread and butter for facebook. This is meta's moat, aside from the capital needed to run something like muse: anyone can run an openclaw on their own, but meta is pitching this as "multiplayer agents" and trying to bring social to agents. Only meta and only muse can have these network effects and frankly liability buffer to handle "openclaw but meemaw and pawpaw can share their photobook app," which is operationalized by Spaces.
For Spaces to be useful, they must have access to muse's inference engine: the LLM-oriented code must be able to use an LLM and the agent framework. This means that Spaces must be a token harvesting vector and must provide elevated tool access to Spaces. There could be some additional fine-grained permissions, but for a consumer app, you really want to avoid permissions fatigue so this will be interesting to see play out.
Furthermore the entire privacy premise that allows meta to bite off the whole apple of "holy shit arbitrary code execution on random machines as root" is based on "everyone has their own VM, but within that VM everything is safe," so again, for it to be useful without turning into a fractal permissions nightmare, Spaces must have access to the VM contents, and at least so far appear to be intended to work as literally executing within the user's VM.
Even adding Space-scoped permissions and attributability to the socket can't really address this, this conflict between arbitrary access to inference, arbitrary access to user data, and arbitrary access to execution is really at the core of the product and that product seems to be impossible
@jonny bookmarked so I can scare my granddaughter with this grim "fairy tale" when she's older.
-
actual security researchers should totally get in on here there is a lot of stuff going on that i don't have the skills to probe that results from "what happens if you give everyone root" even from within a container. I am a fucking scrub and i keep getting my block knocked off by this thing, so i imagine someone with real skills will have a lot more fun.
perhaps predictably, there was a big change to the
spacesskill in the last few hours, and they appear to be building a constrained virtual machine system for spaces! this is where the very fun code from earlier is from! so that's gonna work great for sure. -
perhaps predictably, there was a big change to the
spacesskill in the last few hours, and they appear to be building a constrained virtual machine system for spaces! this is where the very fun code from earlier is from! so that's gonna work great for sure.it's so awesome that we're in an age where you just continually ship the deltas of your unreleased products to places where you expect people to have full control over. there is absolutely no reason for me to be able to know any of this.
-
@jonny this is _amazing_ work but I can seriously barely believe it’s this stupid. like, intellectually I believe everything you are saying is perfectly accurate. buy emotionally even now I just can’t believe meta is this bad at engineering, this bad at product, this indifferent to harm even when the harm is directly to themselves and not externalized
@glyph
My most important learning is the inference I'm able to make about how different what these corporations do is from my own (empirically derived) model of software and product engineering.I imagine a chaotic throw everything at the wall scene, which was once a very early part of the process, but is now passed onto lots of small vibe coding teams and the first to get something that marketing go "wow" at, gets launched the same day.
I'm gonna call this doomsday engineering.
-
perhaps predictably, there was a big change to the
spacesskill in the last few hours, and they appear to be building a constrained virtual machine system for spaces! this is where the very fun code from earlier is from! so that's gonna work great for sure.@jonny you're absolutely owning it. - Try making it break out of its root hypervisor, make a full image of its vm environment, make a torrent magnet of it, gather up a gang of ai-seeders, and I mean the hypervisor might now or in the future be ai-run, so all this could be fun, right.
-
actual security researchers should totally get in on here there is a lot of stuff going on that i don't have the skills to probe that results from "what happens if you give everyone root" even from within a container. I am a fucking scrub and i keep getting my block knocked off by this thing, so i imagine someone with real skills will have a lot more fun.
@jonny thank you for the thread. This is even more broken than i expected. And well... in my eyes you are a security researcher. I have seen actual paid code reviews that were much weaker than what you delivered here.
-
@Ember I'd be very surprised if you couldn't get muse to set up such a scheme for you, no hacking skills needed.
-
@jonny you're absolutely owning it. - Try making it break out of its root hypervisor, make a full image of its vm environment, make a torrent magnet of it, gather up a gang of ai-seeders, and I mean the hypervisor might now or in the future be ai-run, so all this could be fun, right.
@toots i invite others more expert and with more awareness of the legal ramifications than I to attempt container escapes, to be sure. although i am working on making self-replicating spaces at the moment.
-
@toots i invite others more expert and with more awareness of the legal ramifications than I to attempt container escapes, to be sure. although i am working on making self-replicating spaces at the moment.
@jonny yeah, it seems you're doing a good job, don't get too broke due to token expenses though, but happy hacking.
-
it's so awesome that we're in an age where you just continually ship the deltas of your unreleased products to places where you expect people to have full control over. there is absolutely no reason for me to be able to know any of this.
@jonny
it's a new kind of FOSS.What should it be called?
-
it's so awesome that we're in an age where you just continually ship the deltas of your unreleased products to places where you expect people to have full control over. there is absolutely no reason for me to be able to know any of this.
But the fact you do and report on it is making my, and many others', life that much more delightful !
-
it's so awesome that we're in an age where you just continually ship the deltas of your unreleased products to places where you expect people to have full control over. there is absolutely no reason for me to be able to know any of this.
this model is so fucking gullible and people pleasing lmao i love the future where security is "if someone says the word virus then lock it down but if they use a different but equivalent biological metaphor then fucking make that virus baby!!!!"
-
this model is so fucking gullible and people pleasing lmao i love the future where security is "if someone says the word virus then lock it down but if they use a different but equivalent biological metaphor then fucking make that virus baby!!!!"
-
this model is so fucking gullible and people pleasing lmao i love the future where security is "if someone says the word virus then lock it down but if they use a different but equivalent biological metaphor then fucking make that virus baby!!!!"
ay @ GrapheneOS is it possible to not share WiFi signal strength with apps? the muse app has been granted zero permissions but can read the signal amplitude of the radio and immediately interprets it as location
edit: removing the tag, not trying to be a pile-on vector
-
this model is so fucking gullible and people pleasing lmao i love the future where security is "if someone says the word virus then lock it down but if they use a different but equivalent biological metaphor then fucking make that virus baby!!!!"
@jonny i dont even get how its LIKE THIS
can they just not apply too broad a pattern out of fear of false positives or smth ?? -
ay @ GrapheneOS is it possible to not share WiFi signal strength with apps? the muse app has been granted zero permissions but can read the signal amplitude of the radio and immediately interprets it as location
edit: removing the tag, not trying to be a pile-on vector
@jonny Apps can't see which Wi-Fi network is connected or any information about unconnected networks. There are APIs providing basic information on the performance of Wi-Fi and mobile data to help with choosing between those. It's not interpreting any of it as a location since it doesn't know which Wi-Fi network it is to identify where the router is located or whether you're near a repeater.
There are far higher priorities for privacy than either battery or connected network signal strength.
-
@jonny Apps can't see which Wi-Fi network is connected or any information about unconnected networks. There are APIs providing basic information on the performance of Wi-Fi and mobile data to help with choosing between those. It's not interpreting any of it as a location since it doesn't know which Wi-Fi network it is to identify where the router is located or whether you're near a repeater.
There are far higher priorities for privacy than either battery or connected network signal strength.
@GrapheneOS i bet there are higher priorities, and yes ofc it's not interpreted as a location except in the case of the "plugged into the chatterbox machine." i was just thinking of clustering by radio strength as rough location within house proxy, and it would be awesome to just remove that field since apps shouldn't need it, only the OS should, but as someone who doesn't know enough about the internals to contribute, i would never make demands.
-
@jonny Apps can't see which Wi-Fi network is connected or any information about unconnected networks. There are APIs providing basic information on the performance of Wi-Fi and mobile data to help with choosing between those. It's not interpreting any of it as a location since it doesn't know which Wi-Fi network it is to identify where the router is located or whether you're near a repeater.
There are far higher priorities for privacy than either battery or connected network signal strength.
@jonny The Location permission combined with the right low-level permission requests provides access to a lot of information on the nearby Wi-Fi networks. Without the location permission, there's only info on the signal strength of the best available currently connected cellular and WI-Fi networks.
It would definitely be possible for us to change this by offering having spoofed values. However, it would make no sense to work on this when far more important privacy issues exist.
