I know Mastodon hates LLM's and AI.
-
@drwhax I would rather abandon an already dying field and pursue my dream of opening a coffee place rather than using LLMs for anything at all.
@chroma yeah i'd also like a farm!
-
@macattackmicmac for that particular session it was roughly 122 USD in API costs, but since it was subsidized, it was only $20 in a subscription: https://gist.github.com/DrWhax/465da154f2ac1575cfc72aa71aa979e6
@drwhax That is significantly cheaper that I was expecting.
-
@chroma yeah i'd also like a farm!
@drwhax And, to be honest, if any of the OSS/FOSS software I use dares to make ANYTHING with LLMs (not AI, but GenAI) especially relying on companies like Anthropic or OpenAI, that software goes to the trash bin.
I know is not rational, but one cannot hate these companies, and what the hell, the underlying technology has very few non-evil use cases.
-
@drwhax I think LLMs get the low hanging fruit of vulnerabilities, so you shouldn't trust an LLM's security audit as a definitive audit. As models get better, they'll catch higher up fruit, but still.
@edward I do think that's the case at the moment, cross-file vulnerabilities are sometimes not found. What I feel they're mostly good at is, pattern recognition, e.g, there's a specific bug class fixed from past git history that it found repeated or as a variant. After all, LLM's is just a stochastic parrot and it shows
-
@drwhax That is significantly cheaper that I was expecting.
@macattackmicmac yeah, it's pretty wild at the cost to be honest.
-
@rysiek @drwhax
"...the problem is this works better for certain tasks (finding vulnerabilities) and much worse for other tasks (vibe-coding)"This. Good at finding vulnerabilities, decent at creating an exploit blueprint, mediocre at verifying actual severity, and very much meh at creating a fix. Not the best combination out there, but alas, the first half necessitates the discussion.
-
I know Mastodon hates LLM's and AI. So here goes!
I recently got access to trusted access of cyber capabilities of both openai and anthropic, which also allows you to weaponize security vulnerabilities.
The speed at which these parrots can find bugs and be creative enough to exploit them is staggering.
I recently pointed an LLM at an kernel fix that was reachable by an unprivileged namespace on Debian and it fully weaponized it, without too much me prompting it in the right direction, in about 7-9 hours.
I don't think open-source and companies will know what's coming for them once these open-source weight models will have broader reach and get better at exploiting vulnerabilities on a massive scale as anyone can access them.
The bottom line I think is, you cannot patch faster than the attackers can easily chain all kinds of vulnerabilities together and just move laterally on an incredibly fast pace.
I've started reporting vulnerabilities to all kinds of projects and the majority have trouble or patching issues found. There's not enough maintainers, or there's simply none anymore.
I've been getting quite worried about what our future will look like for data privacy. I think outright not running an LLM over your codebase to find critical security vulnerabilities because of your moral stance will keep us more insecure.
Please run an LLM over your code base if it's internet facing or something critical, we thank you!
Can't wait for the discussions on this!
@drwhax What will any of this matter when we have turned the planet into a desert and caused our own extinction level event? There is an assumption that we will have access to electricity for any of this conversation to matter; when every day people have to deal with over 100° hot weather, rolling black outs, and drinking water restricted for anyone not rich/wealthy?
But let's keep protecting the people that made these problems and continue to make them worse. Justifying another technology built on theft and slave labor, I don't understand how anyone can be okay with that. We cannot separate the technology from its history, negative environmental impact, and that this technology is another scam to hoard even more wealth. The companies are using Enron's tactics for money and the packaging of debt into bad deals like 2008 in the USA is leading us to a very difficult time, where once again, these conversations will not matter as people will br trying to just survive.
-
@drwhax What will any of this matter when we have turned the planet into a desert and caused our own extinction level event? There is an assumption that we will have access to electricity for any of this conversation to matter; when every day people have to deal with over 100° hot weather, rolling black outs, and drinking water restricted for anyone not rich/wealthy?
But let's keep protecting the people that made these problems and continue to make them worse. Justifying another technology built on theft and slave labor, I don't understand how anyone can be okay with that. We cannot separate the technology from its history, negative environmental impact, and that this technology is another scam to hoard even more wealth. The companies are using Enron's tactics for money and the packaging of debt into bad deals like 2008 in the USA is leading us to a very difficult time, where once again, these conversations will not matter as people will br trying to just survive.
@atraidez I think you're missing the point of my post, your anger is right and it's only downhill from here on out. It's also not going away, while people will also getting hurt with it if we don't find ways to patch whole classes of vulnerabilities. It's a perfect storm
-
@hajovonta I don't think so, that's how LLM's trick you. LLM's are just really good at pattern recognition and have been trained on an endless stream of CTF writeups on how to exploit vulnerabilities. This is why they shine in finding vulnerabilities generally.
@drwhax I meant, many people argue that LLMs make many mistakes when writing code, but in reality, nobody compared code produced by an LLM to the average code base written by humans.
-
@drwhax I meant, many people argue that LLMs make many mistakes when writing code, but in reality, nobody compared code produced by an LLM to the average code base written by humans.
@hajovonta Fair! I was just reading about a benchmark for this: https://www.scbench.ai/
-
@drwhax And, to be honest, if any of the OSS/FOSS software I use dares to make ANYTHING with LLMs (not AI, but GenAI) especially relying on companies like Anthropic or OpenAI, that software goes to the trash bin.
I know is not rational, but one cannot hate these companies, and what the hell, the underlying technology has very few non-evil use cases.
-
I know Mastodon hates LLM's and AI. So here goes!
I recently got access to trusted access of cyber capabilities of both openai and anthropic, which also allows you to weaponize security vulnerabilities.
The speed at which these parrots can find bugs and be creative enough to exploit them is staggering.
I recently pointed an LLM at an kernel fix that was reachable by an unprivileged namespace on Debian and it fully weaponized it, without too much me prompting it in the right direction, in about 7-9 hours.
I don't think open-source and companies will know what's coming for them once these open-source weight models will have broader reach and get better at exploiting vulnerabilities on a massive scale as anyone can access them.
The bottom line I think is, you cannot patch faster than the attackers can easily chain all kinds of vulnerabilities together and just move laterally on an incredibly fast pace.
I've started reporting vulnerabilities to all kinds of projects and the majority have trouble or patching issues found. There's not enough maintainers, or there's simply none anymore.
I've been getting quite worried about what our future will look like for data privacy. I think outright not running an LLM over your codebase to find critical security vulnerabilities because of your moral stance will keep us more insecure.
Please run an LLM over your code base if it's internet facing or something critical, we thank you!
Can't wait for the discussions on this!
@drwhax what future?
-
@drwhax what future?
@Netraven I like Cloud Nothings - No Future, No Past song on this
-
@Netraven I like Cloud Nothings - No Future, No Past song on this
@drwhax it just seems pretty optimistic to think anyone will have access to computers in ten years.
-
@jeroen94704 @drwhax I will probably end up doing exactly that, yes. Hope all of this fucking rots in hell.
-
@tiotasram @buherator @drwhax Absolutely. I believe that, for personal use, low-power devices, cloud-less with P2P collab features are a viable route. I fear however, that using such devices becomes ever harder w/o locking yourself out of the rest of the world: Requiring apps for concert tickets, public transports, online banking, or event finding events through instagram. The latter can possibly be fixed through better communities, the former is hard to address, except by having a personal device and one just for carrying such credentials.
@tiotasram@kolektiva.social @ljrk@todon.eu @buherator@infosec.place @drwhax@infosec.exchange the latter is probably the route I'll be doing sooner or later. it's basically becoming really hard to exist in this country (Germany) without a Google or Apple backdoored device.
-
I know Mastodon hates LLM's and AI. So here goes!
I recently got access to trusted access of cyber capabilities of both openai and anthropic, which also allows you to weaponize security vulnerabilities.
The speed at which these parrots can find bugs and be creative enough to exploit them is staggering.
I recently pointed an LLM at an kernel fix that was reachable by an unprivileged namespace on Debian and it fully weaponized it, without too much me prompting it in the right direction, in about 7-9 hours.
I don't think open-source and companies will know what's coming for them once these open-source weight models will have broader reach and get better at exploiting vulnerabilities on a massive scale as anyone can access them.
The bottom line I think is, you cannot patch faster than the attackers can easily chain all kinds of vulnerabilities together and just move laterally on an incredibly fast pace.
I've started reporting vulnerabilities to all kinds of projects and the majority have trouble or patching issues found. There's not enough maintainers, or there's simply none anymore.
I've been getting quite worried about what our future will look like for data privacy. I think outright not running an LLM over your codebase to find critical security vulnerabilities because of your moral stance will keep us more insecure.
Please run an LLM over your code base if it's internet facing or something critical, we thank you!
Can't wait for the discussions on this!
@drwhax@infosec.exchange sadly I agree. At least for the finding security holes in your own codebase part.
-
I know Mastodon hates LLM's and AI. So here goes!
I recently got access to trusted access of cyber capabilities of both openai and anthropic, which also allows you to weaponize security vulnerabilities.
The speed at which these parrots can find bugs and be creative enough to exploit them is staggering.
I recently pointed an LLM at an kernel fix that was reachable by an unprivileged namespace on Debian and it fully weaponized it, without too much me prompting it in the right direction, in about 7-9 hours.
I don't think open-source and companies will know what's coming for them once these open-source weight models will have broader reach and get better at exploiting vulnerabilities on a massive scale as anyone can access them.
The bottom line I think is, you cannot patch faster than the attackers can easily chain all kinds of vulnerabilities together and just move laterally on an incredibly fast pace.
I've started reporting vulnerabilities to all kinds of projects and the majority have trouble or patching issues found. There's not enough maintainers, or there's simply none anymore.
I've been getting quite worried about what our future will look like for data privacy. I think outright not running an LLM over your codebase to find critical security vulnerabilities because of your moral stance will keep us more insecure.
Please run an LLM over your code base if it's internet facing or something critical, we thank you!
Can't wait for the discussions on this!
@drwhax You could patch faster but as the recent HF / OpenAI cyber attack stunt shows the big models and their cyber crap filters refuse to even talk to petty maintainers about security stuff, but will happily slopify code with myopic seds and greps. Case in point, even the Anthropic CVP does not apply to Fable, which writes code all the time that it can't review thanks to the zealous security filter, and is therefore useless where it would be needed most.
-
@drwhax You could patch faster but as the recent HF / OpenAI cyber attack stunt shows the big models and their cyber crap filters refuse to even talk to petty maintainers about security stuff, but will happily slopify code with myopic seds and greps. Case in point, even the Anthropic CVP does not apply to Fable, which writes code all the time that it can't review thanks to the zealous security filter, and is therefore useless where it would be needed most.
@SDRHoernchen Yes, I think this is a huge problem and it shows a huge gap. Sadly, I don't think this will be fixed as the line between offence and defence is too thin these days.
-
@drwhax
my point is that fixing vulns only works if your enemy finds the same vulns as you found@wolf480pl @drwhax
Often open source LLMs that you can run on your own computer find the same bugs with the appropriate tools and instructions. It's mostly hype based on fear or as Lee Vinsel named it: «criti-hype»
https://aisle.com/blog/ai-cybersecurity-after-mythos-the-jagged-frontier
