I know Mastodon hates LLM's and AI.
-
@drwhax It's not that you're wrong but you're not specific enough. Which AI, in what way? Plus, though I see you have enough technical expertise you come out with that OpenAI/Claude vibe that I detest. We should use LLMs, but in our own way, after stripping away the hype. All the LLM technobros have been stealing our code and other people's copyrighted work, and now want to sell it back to us with added bias, lack of provenance and old data. Excuse me if I'm not wildly enthusiastic about their for-profit products.
@fnrd you're right, but I think if we take this defeatist stance we're not going to improve things for the better.
Unfortunately, we'll not be able to make our own models, we're compute starved, our best bet is maybe open-weights models.
It's all a mess though, I do agree with that.
-
The nice thing about open source is that there's always *someone* doing whatever obscure thing the mainstream finds too niche. I'm sure there are people right now who have for decades been building smaller software, and maybe for a certain non-corporate group there can be a pivot to that stuff. To avoid an endless edge of new code to harden, you need to forgo compatibility with new software & hardware, which has until now meant relative uselessness. But given how the AI bubble is driving up hardware prices, and the fact that we've seen standards like USB mature to a point that feels like it could be stable for a decade or more, maybe the endless cycle of new hardware stuff just slows down in certain backwaters. Of course this is anathema to corporate strategies, but I wonder if given the fact that the churning edge will be nearly impossible to secure, maybe a chunk of consumers will find ways to be satisfied with a smaller more stable core.
@tiotasram @buherator @drwhax Absolutely. I believe that, for personal use, low-power devices, cloud-less with P2P collab features are a viable route. I fear however, that using such devices becomes ever harder w/o locking yourself out of the rest of the world: Requiring apps for concert tickets, public transports, online banking, or event finding events through instagram. The latter can possibly be fixed through better communities, the former is hard to address, except by having a personal device and one just for carrying such credentials.
-
I know Mastodon hates LLM's and AI. So here goes!
I recently got access to trusted access of cyber capabilities of both openai and anthropic, which also allows you to weaponize security vulnerabilities.
The speed at which these parrots can find bugs and be creative enough to exploit them is staggering.
I recently pointed an LLM at an kernel fix that was reachable by an unprivileged namespace on Debian and it fully weaponized it, without too much me prompting it in the right direction, in about 7-9 hours.
I don't think open-source and companies will know what's coming for them once these open-source weight models will have broader reach and get better at exploiting vulnerabilities on a massive scale as anyone can access them.
The bottom line I think is, you cannot patch faster than the attackers can easily chain all kinds of vulnerabilities together and just move laterally on an incredibly fast pace.
I've started reporting vulnerabilities to all kinds of projects and the majority have trouble or patching issues found. There's not enough maintainers, or there's simply none anymore.
I've been getting quite worried about what our future will look like for data privacy. I think outright not running an LLM over your codebase to find critical security vulnerabilities because of your moral stance will keep us more insecure.
Please run an LLM over your code base if it's internet facing or something critical, we thank you!
Can't wait for the discussions on this!
@drwhax The laws of diminishing returns will kick in at some point. In the meantime, it is unlikely to be a smooth ride.
-
@drwhax
isn't this a proof that humans are not better at writing software than SOTA models? In general, of course.@hajovonta I don't think so, that's how LLM's trick you. LLM's are just really good at pattern recognition and have been trained on an endless stream of CTF writeups on how to exploit vulnerabilities. This is why they shine in finding vulnerabilities generally.
-
@drwhax The laws of diminishing returns will kick in at some point. In the meantime, it is unlikely to be a smooth ride.
@whvholst it's one crazy rollercoaster right, including after the subsidy for compute ends
-
I know Mastodon hates LLM's and AI. So here goes!
I recently got access to trusted access of cyber capabilities of both openai and anthropic, which also allows you to weaponize security vulnerabilities.
The speed at which these parrots can find bugs and be creative enough to exploit them is staggering.
I recently pointed an LLM at an kernel fix that was reachable by an unprivileged namespace on Debian and it fully weaponized it, without too much me prompting it in the right direction, in about 7-9 hours.
I don't think open-source and companies will know what's coming for them once these open-source weight models will have broader reach and get better at exploiting vulnerabilities on a massive scale as anyone can access them.
The bottom line I think is, you cannot patch faster than the attackers can easily chain all kinds of vulnerabilities together and just move laterally on an incredibly fast pace.
I've started reporting vulnerabilities to all kinds of projects and the majority have trouble or patching issues found. There's not enough maintainers, or there's simply none anymore.
I've been getting quite worried about what our future will look like for data privacy. I think outright not running an LLM over your codebase to find critical security vulnerabilities because of your moral stance will keep us more insecure.
Please run an LLM over your code base if it's internet facing or something critical, we thank you!
Can't wait for the discussions on this!
@drwhax Look, many people have different reasons to be against AI.
Mine is a very specific position against vibe coding, generative AI and ghoulish tech bros and their wannabes wanting to actively destroy excellence, art, nature and human work.
If you have a tool that does not contradict these specifics, then I have no issue. A static analyser ML-like code tool that works on premises, doesn't make a tech bro richer and dry a lake? This is OK for me (and I suspect, the future or much of this)
-
1. Acceptance. There's the saying "you just need to be more secure than your neighbor". It's not fully true but biz people hold it so. It's flip side is a cynic commentary on security, and a true one at that: If your neighbor is not secure, you don't need to be either. This is still true if you're getting pwned. When everyone's getting hacked regularly, society will accept this new reality. Only the defense sector will still care for security. We've seen similar developments in different industries: Fast fashion isn't an adequate replacement for garment that stands the test of time. But it's getting impossible to buy good clothes, even if you want to. But also with other goods: If everyone produces low quality items, it's financially more reasonable to do the same.
2. Centralization. Coz we won't do the right thing and reduce complexity, we need to find a way to make complexity "secure": Cloud, Big5, etc. Corps that have the resources to host an Exchange server securely... or in compliance with regulations. Every tool you do not let a big house host is a liability. Not coz they're necessarily more secure, but because nobody will blame you if something gets wrong. We had that with "buying IBM" once.
I hate this outlook but I seriously doubt that we will correct course.
@ljrk @drwhax Re: 1., the usual question we got from CISO and above on our reports is "how does it look compared to similar companies?". I agree that this is in part the "you don't have to outrun the bear" logic, but also that people in position don't want to look incompetent in front of their peers.
Another thing is that you don't get fired if you didn't follow the hackers advice, but you do get fired if you don't pass compliance which is one part BS, and the other part is easy to cheat. -
@drwhax Look, many people have different reasons to be against AI.
Mine is a very specific position against vibe coding, generative AI and ghoulish tech bros and their wannabes wanting to actively destroy excellence, art, nature and human work.
If you have a tool that does not contradict these specifics, then I have no issue. A static analyser ML-like code tool that works on premises, doesn't make a tech bro richer and dry a lake? This is OK for me (and I suspect, the future or much of this)
@drwhax But are you maling a tech bro richer to vibe code everything you do and just pressing Y without thinking, wasting tokens as much as possible because who cares? I have beef.
Are you actively (and disgustingly so), trying to replace every human in the loop just to create ghibli styled art slop or disgusting political videos so you can further your agenda? I have beef.
Are you actively numbing your brain, stop thinking about anything and becoming a reverse centaur on purpose? I have beef.
-
@whvholst it's one crazy rollercoaster right, including after the subsidy for compute ends
@drwhax On the flip side, when the subsidy for compute ends, edge computing will become affordable again. And given that things like rudimentary voice recognition now has been reduced to models that fit in an ESP32, I am inclined to think that special-purpose LLMs may be a cat that has left the bag or at least to be in a different league than frontier models.
-
@drwhax On the flip side, when the subsidy for compute ends, edge computing will become affordable again. And given that things like rudimentary voice recognition now has been reduced to models that fit in an ESP32, I am inclined to think that special-purpose LLMs may be a cat that has left the bag or at least to be in a different league than frontier models.
@whvholst maybe things will turn cheap(er) again? I have some doubts about this and there's all kinds of ways to keep prices for components high by the companies and they've done this before and will do it again and they seem to get away with it mostly.
-
I know Mastodon hates LLM's and AI. So here goes!
I recently got access to trusted access of cyber capabilities of both openai and anthropic, which also allows you to weaponize security vulnerabilities.
The speed at which these parrots can find bugs and be creative enough to exploit them is staggering.
I recently pointed an LLM at an kernel fix that was reachable by an unprivileged namespace on Debian and it fully weaponized it, without too much me prompting it in the right direction, in about 7-9 hours.
I don't think open-source and companies will know what's coming for them once these open-source weight models will have broader reach and get better at exploiting vulnerabilities on a massive scale as anyone can access them.
The bottom line I think is, you cannot patch faster than the attackers can easily chain all kinds of vulnerabilities together and just move laterally on an incredibly fast pace.
I've started reporting vulnerabilities to all kinds of projects and the majority have trouble or patching issues found. There's not enough maintainers, or there's simply none anymore.
I've been getting quite worried about what our future will look like for data privacy. I think outright not running an LLM over your codebase to find critical security vulnerabilities because of your moral stance will keep us more insecure.
Please run an LLM over your code base if it's internet facing or something critical, we thank you!
Can't wait for the discussions on this!
> I think outright not running an LLM over your codebase to find critical security vulnerabilities because of your moral stance will keep us more insecure
You're missing the point. Even strong "AI critics" have already been seen to use these tools for these scans themselves. Just like other static code analysis tools and so on.
The main point and what people despise the most is when randoms use it and file bug reports they don't understand causing high workload validating their crap.
-
@drwhax But are you maling a tech bro richer to vibe code everything you do and just pressing Y without thinking, wasting tokens as much as possible because who cares? I have beef.
Are you actively (and disgustingly so), trying to replace every human in the loop just to create ghibli styled art slop or disgusting political videos so you can further your agenda? I have beef.
Are you actively numbing your brain, stop thinking about anything and becoming a reverse centaur on purpose? I have beef.
@drwhax I personally dont and will never vibe code because, in a very personal position, I love my brain and I like to think, and a brain without friction will cease to function (use it or lose it).
I love people making art (and yes, this includes code). And I deeply hate the contempt for people paying a tech bro to use genAI to dismiss and degrade those artists with a sort of vindictive glee, especially since most artists are already treated so badly. It is disgusting.
-
@drwhax I personally dont and will never vibe code because, in a very personal position, I love my brain and I like to think, and a brain without friction will cease to function (use it or lose it).
I love people making art (and yes, this includes code). And I deeply hate the contempt for people paying a tech bro to use genAI to dismiss and degrade those artists with a sort of vindictive glee, especially since most artists are already treated so badly. It is disgusting.
@glitchypixel I think art should be made by artists and not AI fwiw!
-
> I think outright not running an LLM over your codebase to find critical security vulnerabilities because of your moral stance will keep us more insecure
You're missing the point. Even strong "AI critics" have already been seen to use these tools for these scans themselves. Just like other static code analysis tools and so on.
The main point and what people despise the most is when randoms use it and file bug reports they don't understand causing high workload validating their crap.
@agowa338 I agree and that's unfortunately the shitty side of it.
-
@drwhax I personally dont and will never vibe code because, in a very personal position, I love my brain and I like to think, and a brain without friction will cease to function (use it or lose it).
I love people making art (and yes, this includes code). And I deeply hate the contempt for people paying a tech bro to use genAI to dismiss and degrade those artists with a sort of vindictive glee, especially since most artists are already treated so badly. It is disgusting.
@drwhax But I don't mind the underlying tech as much as I dont have a beef against the, let's say Microsoft Kinect (a tool made in essence the same way).
So are you getting a Chinese open weight, AI (people won't say it, maybe because of fear of retaliation from the US, but most companies will do this) stuffing it in your server and having 90% effectiveness to find vulnerabilities and doing static analysis in code locally?
That sounds like an actual use case. Just one the tech bros don't like.
-
I know Mastodon hates LLM's and AI. So here goes!
I recently got access to trusted access of cyber capabilities of both openai and anthropic, which also allows you to weaponize security vulnerabilities.
The speed at which these parrots can find bugs and be creative enough to exploit them is staggering.
I recently pointed an LLM at an kernel fix that was reachable by an unprivileged namespace on Debian and it fully weaponized it, without too much me prompting it in the right direction, in about 7-9 hours.
I don't think open-source and companies will know what's coming for them once these open-source weight models will have broader reach and get better at exploiting vulnerabilities on a massive scale as anyone can access them.
The bottom line I think is, you cannot patch faster than the attackers can easily chain all kinds of vulnerabilities together and just move laterally on an incredibly fast pace.
I've started reporting vulnerabilities to all kinds of projects and the majority have trouble or patching issues found. There's not enough maintainers, or there's simply none anymore.
I've been getting quite worried about what our future will look like for data privacy. I think outright not running an LLM over your codebase to find critical security vulnerabilities because of your moral stance will keep us more insecure.
Please run an LLM over your code base if it's internet facing or something critical, we thank you!
Can't wait for the discussions on this!
@drwhax
do two differemt LLMs find the same bugs? -
@ljrk @drwhax Re: 1., the usual question we got from CISO and above on our reports is "how does it look compared to similar companies?". I agree that this is in part the "you don't have to outrun the bear" logic, but also that people in position don't want to look incompetent in front of their peers.
Another thing is that you don't get fired if you didn't follow the hackers advice, but you do get fired if you don't pass compliance which is one part BS, and the other part is easy to cheat.@buherator @drwhax Yup same. It's what we do at $dayjob to motivated customers, partially. I'm waiting for a cohort to realize that, if nobody does anything, nobody will look too bad.
And yup. It's all a bit fuked up. :3
-
@whvholst maybe things will turn cheap(er) again? I have some doubts about this and there's all kinds of ways to keep prices for components high by the companies and they've done this before and will do it again and they seem to get away with it mostly.
@drwhax I am old enough to remember RAM prices fluctuating between "having to sell a kidney" and "oooh, I get to max out my motherboard if I collect the deposit on these empty beer bottles" several times.
-
@drwhax
do two differemt LLMs find the same bugs?@wolf480pl sometimes, sometimes they don't, sometimes one finds a better way to chain vulnerabilities to achieve a certain objective. It all depends a bit on the harness as well, lots of small knobs to twist.
Some benchmarks are available on: https://exploitbench.ai/
-
@wolf480pl sometimes, sometimes they don't, sometimes one finds a better way to chain vulnerabilities to achieve a certain objective. It all depends a bit on the harness as well, lots of small knobs to twist.
Some benchmarks are available on: https://exploitbench.ai/
@drwhax
my point is that fixing vulns only works if your enemy finds the same vulns as you found
