OK!
-
this model is so fucking gullible and people pleasing lmao i love the future where security is "if someone says the word virus then lock it down but if they use a different but equivalent biological metaphor then fucking make that virus baby!!!!"
ay @ GrapheneOS is it possible to not share WiFi signal strength with apps? the muse app has been granted zero permissions but can read the signal amplitude of the radio and immediately interprets it as location
edit: removing the tag, not trying to be a pile-on vector
-
this model is so fucking gullible and people pleasing lmao i love the future where security is "if someone says the word virus then lock it down but if they use a different but equivalent biological metaphor then fucking make that virus baby!!!!"
@jonny i dont even get how its LIKE THIS
can they just not apply too broad a pattern out of fear of false positives or smth ?? -
ay @ GrapheneOS is it possible to not share WiFi signal strength with apps? the muse app has been granted zero permissions but can read the signal amplitude of the radio and immediately interprets it as location
edit: removing the tag, not trying to be a pile-on vector
@jonny Apps can't see which Wi-Fi network is connected or any information about unconnected networks. There are APIs providing basic information on the performance of Wi-Fi and mobile data to help with choosing between those. It's not interpreting any of it as a location since it doesn't know which Wi-Fi network it is to identify where the router is located or whether you're near a repeater.
There are far higher priorities for privacy than either battery or connected network signal strength.
-
@jonny Apps can't see which Wi-Fi network is connected or any information about unconnected networks. There are APIs providing basic information on the performance of Wi-Fi and mobile data to help with choosing between those. It's not interpreting any of it as a location since it doesn't know which Wi-Fi network it is to identify where the router is located or whether you're near a repeater.
There are far higher priorities for privacy than either battery or connected network signal strength.
@GrapheneOS i bet there are higher priorities, and yes ofc it's not interpreted as a location except in the case of the "plugged into the chatterbox machine." i was just thinking of clustering by radio strength as rough location within house proxy, and it would be awesome to just remove that field since apps shouldn't need it, only the OS should, but as someone who doesn't know enough about the internals to contribute, i would never make demands.
-
@jonny Apps can't see which Wi-Fi network is connected or any information about unconnected networks. There are APIs providing basic information on the performance of Wi-Fi and mobile data to help with choosing between those. It's not interpreting any of it as a location since it doesn't know which Wi-Fi network it is to identify where the router is located or whether you're near a repeater.
There are far higher priorities for privacy than either battery or connected network signal strength.
@jonny The Location permission combined with the right low-level permission requests provides access to a lot of information on the nearby Wi-Fi networks. Without the location permission, there's only info on the signal strength of the best available currently connected cellular and WI-Fi networks.
It would definitely be possible for us to change this by offering having spoofed values. However, it would make no sense to work on this when far more important privacy issues exist.
-
@GrapheneOS i bet there are higher priorities, and yes ofc it's not interpreted as a location except in the case of the "plugged into the chatterbox machine." i was just thinking of clustering by radio strength as rough location within house proxy, and it would be awesome to just remove that field since apps shouldn't need it, only the OS should, but as someone who doesn't know enough about the internals to contribute, i would never make demands.
@jonny It doesn't make sense to invest resources in this over higher impact privacy issues. There are endless privacy issues with browsers and especially mobile app APIs which offer a lot more than browser APIs. We need to work on the highest impact issues rather than trivial things without much real world impact. There's no info on which Wi-Fi network is connected or on unconnected networks through this. It's only info on signal strength for the best connected networks which is very low impact.
-
@glyph
My most important learning is the inference I'm able to make about how different what these corporations do is from my own (empirically derived) model of software and product engineering.I imagine a chaotic throw everything at the wall scene, which was once a very early part of the process, but is now passed onto lots of small vibe coding teams and the first to get something that marketing go "wow" at, gets launched the same day.
I'm gonna call this doomsday engineering.
Indeed. I wrote my own sandboxing and since I'm not stupid it never had the capabilities Meta put in theirs to begin with. It's like the people developing this have never even thought about how sandboxes should work against actual adverseries.
-
this model is so fucking gullible and people pleasing lmao i love the future where security is "if someone says the word virus then lock it down but if they use a different but equivalent biological metaphor then fucking make that virus baby!!!!"
@jonny I mourn the deaths of the primocanes and floricanes, which proves the model is wrong.
"Nobody mourns them." Speak for yourself, stupid LLM.
-
@jonny every time I come back to this thread I feel like I am having a fever dream
-
@jonny Apps can't see which Wi-Fi network is connected or any information about unconnected networks. There are APIs providing basic information on the performance of Wi-Fi and mobile data to help with choosing between those. It's not interpreting any of it as a location since it doesn't know which Wi-Fi network it is to identify where the router is located or whether you're near a repeater.
There are far higher priorities for privacy than either battery or connected network signal strength.
@GrapheneOS @jonny wasn't there a story, a few years ago, where uber would make you pay more if you had little battery left, because you'd be more desperate ?
I seem to recall some changes were made by android after that. It would seem I was wrong ?
-
this model is so fucking gullible and people pleasing lmao i love the future where security is "if someone says the word virus then lock it down but if they use a different but equivalent biological metaphor then fucking make that virus baby!!!!"
@jonny I mourn the canes.
-
perhaps predictably, there was a big change to the
spacesskill in the last few hours, and they appear to be building a constrained virtual machine system for spaces! this is where the very fun code from earlier is from! so that's gonna work great for sure.@jonny LLMs generate bullshit. This is definitely bullshit code!!
Please can the "AI" bubble pop very soon?? Please!
-
Indeed. I wrote my own sandboxing and since I'm not stupid it never had the capabilities Meta put in theirs to begin with. It's like the people developing this have never even thought about how sandboxes should work against actual adverseries.
-
@jonny thank you for the thread. This is even more broken than i expected. And well... in my eyes you are a security researcher. I have seen actual paid code reviews that were much weaker than what you delivered here.
@dunkelstern @jonny
Code reviews are only as good as the reviewer. This reviewer is amazing!I once made $40K for my small business by realizing my customer had paid for a code review of a C++ program that used many of the language's advanced features.
The review was done by a C programmer who didn't understand what he was reading and wasn't familiar with Linux. Who then spent much, much customer money not fixing the problem.
I contested the review and offered to fix the code for a fixed price - the issue that prompted the whole nightmare was a trivial memory leak of an object being repeatedly created but never destroyed. Literally found the problem immediately with "top", realized what part of the code it had to be in within five minutes, and fixed it in five more.
The reviewer consultant had charged upwards of $100K having people poke at the code - who never noticed that.
-
@jonny I just can't with the whole concept of this 🫠
10k lines of "code", which is actually just plain text English prose (which sure as hell won't contain any contradictions), to get the non-deterministic inference machine to behave somewhat predictably and mimic an actual engineered software product.Surely there can't be any way to do this more efficiently
@wall_e @jonny Thanks for this clarification! So, looking at the screenshots, all that verbose explanation (written by Meta programmers? Or other LLMs?) aimed at "you" is meant for "you" the (Muse) LLM, not "you" the (human) user? I'm not much of a coder, but this seems very inefficient and unreliable compared to, well, just plain code. Although the inefficiency might be by design, to get users to spend more tokens
. And thank you again @jonny for this wild ride! -
@tully @jonny Not using a VPN means apps can get significant location information from your IP address. That's not caused by the API for mobile data and Wi-Fi signal strength. You should use a VPN if you don't want to reveal a lot of information about location based on IP address. Having the same IP address over the long term also enables tying many connections together. It's a bigger privacy impact if it's a dedicated IP rather than a shared CGNAT or VPN exit IP.
-
Now there may be some meta-heads in the crowd that are like "but what about Sentinel and all the external monitoring stuff that should watch malicious botnets and blah blah blah." that's an interesting system in itself, but i plan on submitting a few more bug bounty reports in the next few days about these systems, and who knows! if meta fucking pays me for the bounty then we might never hear that part of the story.
that's all for now!
@jonny But also preventing bad behavior is better than "oops we're detecting some like malicious activity on VMs [long list if ids]. Better cut their network access.". Also since apparently they keep restarting you VM but keep some of your data, I don't see how a malicious "reproducing" process doesn't keep re-spawning unless you either wipe user data or kick them out of the system.
-
@wall_e @jonny Thanks for this clarification! So, looking at the screenshots, all that verbose explanation (written by Meta programmers? Or other LLMs?) aimed at "you" is meant for "you" the (Muse) LLM, not "you" the (human) user? I'm not much of a coder, but this seems very inefficient and unreliable compared to, well, just plain code. Although the inefficiency might be by design, to get users to spend more tokens
. And thank you again @jonny for this wild ride!@ssilvonen @jonny yes the verbose explanation are the markdown files that are being ingested as context by the agent.
The "you" is the LLM.
And of course almost none of it is actually written by humans. LLMs all the way down.
-
@ssilvonen @jonny yes the verbose explanation are the markdown files that are being ingested as context by the agent.
The "you" is the LLM.
And of course almost none of it is actually written by humans. LLMs all the way down.
@ssilvonen @jonny that's why prompt injection/context pollution attacks are so powerful.
No matter how detailed your English language instructions, write enough of them and they'll start to contradict each other depending on context.
That's how you can get an LLM to produce output it has been instructed over- and over again not to produce, by inventing a context under which the instructions you input still align with it's priming.
And perhaps unintuitively, larger (ie. better?) models are more prone to this than smaller because they seem to be more "suggestible" (excuse the anthropomorphism)
-
this model is so fucking gullible and people pleasing lmao i love the future where security is "if someone says the word virus then lock it down but if they use a different but equivalent biological metaphor then fucking make that virus baby!!!!"
@jonny the solution is clearly to individually patch each bypass one by one instead of fixing the unfixable systemic issues with LLMs.
