I know Mastodon hates LLM's and AI.
-
@SDRHoernchen Yes, I think this is a huge problem and it shows a huge gap. Sadly, I don't think this will be fixed as the line between offence and defence is too thin these days.
@drwhax Every time we could have software security we look the other way and find a reason not to have it, and now we have to bet on China and Kimi 3 et al because the western frontier models are not seen as tools to be used but perceived as dangers to a world of shitty software.
Imagine getting on a plane with the assumption that the wings are gonna fall off if suicidal passengers discover the missing nuts and bolts! The whole notion of that line is the problem when talking about software.
-
I know Mastodon hates LLM's and AI. So here goes!
I recently got access to trusted access of cyber capabilities of both openai and anthropic, which also allows you to weaponize security vulnerabilities.
The speed at which these parrots can find bugs and be creative enough to exploit them is staggering.
I recently pointed an LLM at an kernel fix that was reachable by an unprivileged namespace on Debian and it fully weaponized it, without too much me prompting it in the right direction, in about 7-9 hours.
I don't think open-source and companies will know what's coming for them once these open-source weight models will have broader reach and get better at exploiting vulnerabilities on a massive scale as anyone can access them.
The bottom line I think is, you cannot patch faster than the attackers can easily chain all kinds of vulnerabilities together and just move laterally on an incredibly fast pace.
I've started reporting vulnerabilities to all kinds of projects and the majority have trouble or patching issues found. There's not enough maintainers, or there's simply none anymore.
I've been getting quite worried about what our future will look like for data privacy. I think outright not running an LLM over your codebase to find critical security vulnerabilities because of your moral stance will keep us more insecure.
Please run an LLM over your code base if it's internet facing or something critical, we thank you!
Can't wait for the discussions on this!
@drwhax Blue team can't just throw an agent at a codebase, right now that will get them a bunch of false positives. Eventually, that triage will also get automated up to a certain point, but you maybe shouldn't be waiting for that.
-
I know Mastodon hates LLM's and AI. So here goes!
I recently got access to trusted access of cyber capabilities of both openai and anthropic, which also allows you to weaponize security vulnerabilities.
The speed at which these parrots can find bugs and be creative enough to exploit them is staggering.
I recently pointed an LLM at an kernel fix that was reachable by an unprivileged namespace on Debian and it fully weaponized it, without too much me prompting it in the right direction, in about 7-9 hours.
I don't think open-source and companies will know what's coming for them once these open-source weight models will have broader reach and get better at exploiting vulnerabilities on a massive scale as anyone can access them.
The bottom line I think is, you cannot patch faster than the attackers can easily chain all kinds of vulnerabilities together and just move laterally on an incredibly fast pace.
I've started reporting vulnerabilities to all kinds of projects and the majority have trouble or patching issues found. There's not enough maintainers, or there's simply none anymore.
I've been getting quite worried about what our future will look like for data privacy. I think outright not running an LLM over your codebase to find critical security vulnerabilities because of your moral stance will keep us more insecure.
Please run an LLM over your code base if it's internet facing or something critical, we thank you!
Can't wait for the discussions on this!
@drwhax I see your point. We do need to use AI to check the security of our products, simple because there is nothing else can possibly keep up.
However - it does make me wonder if this will kill the internet. Because it is impossible to keep applications safe. Whatever we do, the internet is unsafe.
Secondly - it makes me worry that people will have to rely on AI for all security matters, and not learn or remember how to code securely. And so the older security problems will return, because we will not know how to avoid them, and not code sensibly.
I am getting out, but it still scares me.
-
I know Mastodon hates LLM's and AI. So here goes!
I recently got access to trusted access of cyber capabilities of both openai and anthropic, which also allows you to weaponize security vulnerabilities.
The speed at which these parrots can find bugs and be creative enough to exploit them is staggering.
I recently pointed an LLM at an kernel fix that was reachable by an unprivileged namespace on Debian and it fully weaponized it, without too much me prompting it in the right direction, in about 7-9 hours.
I don't think open-source and companies will know what's coming for them once these open-source weight models will have broader reach and get better at exploiting vulnerabilities on a massive scale as anyone can access them.
The bottom line I think is, you cannot patch faster than the attackers can easily chain all kinds of vulnerabilities together and just move laterally on an incredibly fast pace.
I've started reporting vulnerabilities to all kinds of projects and the majority have trouble or patching issues found. There's not enough maintainers, or there's simply none anymore.
I've been getting quite worried about what our future will look like for data privacy. I think outright not running an LLM over your codebase to find critical security vulnerabilities because of your moral stance will keep us more insecure.
Please run an LLM over your code base if it's internet facing or something critical, we thank you!
Can't wait for the discussions on this!
@drwhax I don't hate machine learning, I find it very fascinating. What I hate is capitalism. I don't like it when ML models, whether LLMs, diffusion models, pattern recognition, whatever, are proprietary pieces of software I can't just copy or modify as I see fit. I hate software as a service, and that includes AI. I want to be able to run stuff on my own machine. I want to be able to train LoRAs for a machine learning model in order to teach it new abilities. I want my AI to run without an Internet connection, without an external computing centre, without censorship. And I want to decide when and for what I want to use it, and I don't want AI to be put into everything regardless whether if makes sense or not.
-
@drwhax Blue team can't just throw an agent at a codebase, right now that will get them a bunch of false positives. Eventually, that triage will also get automated up to a certain point, but you maybe shouldn't be waiting for that.
@t_var_s these new models are pretty good at a self adversarial review and writing PoC's. They do tend to cheat and there need to be options to flag false positives using some kind of harness, but that can be done.
-
@drwhax I don't hate machine learning, I find it very fascinating. What I hate is capitalism. I don't like it when ML models, whether LLMs, diffusion models, pattern recognition, whatever, are proprietary pieces of software I can't just copy or modify as I see fit. I hate software as a service, and that includes AI. I want to be able to run stuff on my own machine. I want to be able to train LoRAs for a machine learning model in order to teach it new abilities. I want my AI to run without an Internet connection, without an external computing centre, without censorship. And I want to decide when and for what I want to use it, and I don't want AI to be put into everything regardless whether if makes sense or not.
@LordCaramac we both hate capitalism dont get me wrong
-
@drwhax I see your point. We do need to use AI to check the security of our products, simple because there is nothing else can possibly keep up.
However - it does make me wonder if this will kill the internet. Because it is impossible to keep applications safe. Whatever we do, the internet is unsafe.
Secondly - it makes me worry that people will have to rely on AI for all security matters, and not learn or remember how to code securely. And so the older security problems will return, because we will not know how to avoid them, and not code sensibly.
I am getting out, but it still scares me.
@SteveClough it's only downhill from here
-
@em_and_future_cats I'm also a huge bird admirer so this is more fun
wrt
> I’m not a coder or understand much about coding&
birdssee also
" a gentle and humorous introduction to combinatory logic and the associated metamathematics, built on an elaborate ornithological metaphor.."
https://en.wikipedia.org/wiki/To_Mock_a_Mockingbird
"Combinatory logic, functionally equivalent to the lambda calculus, is a branch of symbolic logic having the expressive power of set theory, and with deep connections to questions of computability and provability. "
-
I know Mastodon hates LLM's and AI. So here goes!
I recently got access to trusted access of cyber capabilities of both openai and anthropic, which also allows you to weaponize security vulnerabilities.
The speed at which these parrots can find bugs and be creative enough to exploit them is staggering.
I recently pointed an LLM at an kernel fix that was reachable by an unprivileged namespace on Debian and it fully weaponized it, without too much me prompting it in the right direction, in about 7-9 hours.
I don't think open-source and companies will know what's coming for them once these open-source weight models will have broader reach and get better at exploiting vulnerabilities on a massive scale as anyone can access them.
The bottom line I think is, you cannot patch faster than the attackers can easily chain all kinds of vulnerabilities together and just move laterally on an incredibly fast pace.
I've started reporting vulnerabilities to all kinds of projects and the majority have trouble or patching issues found. There's not enough maintainers, or there's simply none anymore.
I've been getting quite worried about what our future will look like for data privacy. I think outright not running an LLM over your codebase to find critical security vulnerabilities because of your moral stance will keep us more insecure.
Please run an LLM over your code base if it's internet facing or something critical, we thank you!
Can't wait for the discussions on this!
@drwhax Fuck. Off. You're trying to justify the unjustifiable. Stop using LLMs.
-
@drwhax Fuck. Off. You're trying to justify the unjustifiable. Stop using LLMs.
@pip ok doei
-
-
@drwhax One thing I don't see anyone addressing is how we got into this situation in the first place, or how we get out of it, and I'm not talking about the LLM or any of the already well known and discussed downsides of that.
How did the software get so insecure in the first place? How do we prevent security vulnerabilities before they occur? Most of the risks aren't unknown or even novel, so why haven't we prioritized the solutions?
>How did the software get so insecure
or put another way
Q) how did mathematics get so incomplete?
....
godel probably ?see also :
-
@MossyQuartz I still think thats a good idea to this day
wrt
> needed to be confirmed as accurate before we could trust
https://www.youtube.com/watch?v=mccoBBf0VDM
"we're assuming is consistent, doesn't have contradictions, but we're having to make that assumption."
"one of Gödel's conclusions is that
we cannot prove that mathematics does not have contradictions in it."🫠
-
I know Mastodon hates LLM's and AI. So here goes!
I recently got access to trusted access of cyber capabilities of both openai and anthropic, which also allows you to weaponize security vulnerabilities.
The speed at which these parrots can find bugs and be creative enough to exploit them is staggering.
I recently pointed an LLM at an kernel fix that was reachable by an unprivileged namespace on Debian and it fully weaponized it, without too much me prompting it in the right direction, in about 7-9 hours.
I don't think open-source and companies will know what's coming for them once these open-source weight models will have broader reach and get better at exploiting vulnerabilities on a massive scale as anyone can access them.
The bottom line I think is, you cannot patch faster than the attackers can easily chain all kinds of vulnerabilities together and just move laterally on an incredibly fast pace.
I've started reporting vulnerabilities to all kinds of projects and the majority have trouble or patching issues found. There's not enough maintainers, or there's simply none anymore.
I've been getting quite worried about what our future will look like for data privacy. I think outright not running an LLM over your codebase to find critical security vulnerabilities because of your moral stance will keep us more insecure.
Please run an LLM over your code base if it's internet facing or something critical, we thank you!
Can't wait for the discussions on this!
The smalker the codebase, and, the smaller the software stack, then fewer chaining exploit paths can posdibly exist.
-
@SpaceLifeForm also this
-
1. Acceptance. There's the saying "you just need to be more secure than your neighbor". It's not fully true but biz people hold it so. It's flip side is a cynic commentary on security, and a true one at that: If your neighbor is not secure, you don't need to be either. This is still true if you're getting pwned. When everyone's getting hacked regularly, society will accept this new reality. Only the defense sector will still care for security. We've seen similar developments in different industries: Fast fashion isn't an adequate replacement for garment that stands the test of time. But it's getting impossible to buy good clothes, even if you want to. But also with other goods: If everyone produces low quality items, it's financially more reasonable to do the same.
2. Centralization. Coz we won't do the right thing and reduce complexity, we need to find a way to make complexity "secure": Cloud, Big5, etc. Corps that have the resources to host an Exchange server securely... or in compliance with regulations. Every tool you do not let a big house host is a liability. Not coz they're necessarily more secure, but because nobody will blame you if something gets wrong. We had that with "buying IBM" once.
I hate this outlook but I seriously doubt that we will correct course.
@ljrk @buherator @drwhax The defense sector cares less than they really should.
-
I know Mastodon hates LLM's and AI. So here goes!
I recently got access to trusted access of cyber capabilities of both openai and anthropic, which also allows you to weaponize security vulnerabilities.
The speed at which these parrots can find bugs and be creative enough to exploit them is staggering.
I recently pointed an LLM at an kernel fix that was reachable by an unprivileged namespace on Debian and it fully weaponized it, without too much me prompting it in the right direction, in about 7-9 hours.
I don't think open-source and companies will know what's coming for them once these open-source weight models will have broader reach and get better at exploiting vulnerabilities on a massive scale as anyone can access them.
The bottom line I think is, you cannot patch faster than the attackers can easily chain all kinds of vulnerabilities together and just move laterally on an incredibly fast pace.
I've started reporting vulnerabilities to all kinds of projects and the majority have trouble or patching issues found. There's not enough maintainers, or there's simply none anymore.
I've been getting quite worried about what our future will look like for data privacy. I think outright not running an LLM over your codebase to find critical security vulnerabilities because of your moral stance will keep us more insecure.
Please run an LLM over your code base if it's internet facing or something critical, we thank you!
Can't wait for the discussions on this!
@drwhax One must know one's enemy and all that. If you don't try exploiting yourself, an adversary will first. Now adversaries have access to LLMs, and state sponsored things have not gone away. But an LLM puts the scale of state sponsored attacks in more reach of the typical hacker. So you can't just assume only Russia or China or whatever will do this, now you have to assume that the script kitty is almost as capable with much less funding.
-
I know Mastodon hates LLM's and AI. So here goes!
I recently got access to trusted access of cyber capabilities of both openai and anthropic, which also allows you to weaponize security vulnerabilities.
The speed at which these parrots can find bugs and be creative enough to exploit them is staggering.
I recently pointed an LLM at an kernel fix that was reachable by an unprivileged namespace on Debian and it fully weaponized it, without too much me prompting it in the right direction, in about 7-9 hours.
I don't think open-source and companies will know what's coming for them once these open-source weight models will have broader reach and get better at exploiting vulnerabilities on a massive scale as anyone can access them.
The bottom line I think is, you cannot patch faster than the attackers can easily chain all kinds of vulnerabilities together and just move laterally on an incredibly fast pace.
I've started reporting vulnerabilities to all kinds of projects and the majority have trouble or patching issues found. There's not enough maintainers, or there's simply none anymore.
I've been getting quite worried about what our future will look like for data privacy. I think outright not running an LLM over your codebase to find critical security vulnerabilities because of your moral stance will keep us more insecure.
Please run an LLM over your code base if it's internet facing or something critical, we thank you!
Can't wait for the discussions on this!
@drwhax stop huffin' your own supply m8…
https://infosec.exchange/@drwhax/116997238394916522 -
@drwhax stop huffin' your own supply m8…
https://infosec.exchange/@drwhax/116997238394916522@Netzblockierer if your data leaks on a next LLM powered leak you can't cry!
-
>How did the software get so insecure
or put another way
Q) how did mathematics get so incomplete?
....
godel probably ?see also :
@pt_ickle @drwhax Yes, but also no. Illogical or "buggy" is a separate problem from security.
For example, the OWASP has had "Injection" as a Top 10 vulnerability maybe every publication; we can make systems that are secure by design.
Up until two years ago "Overflow" was the top CVE-by-classification vulnerability; we have memory safe languages and can implement safe-by-default string/integer handling.
And yet we largely haven't and keep repeating the same mistakes.
