If you don't trust AI and AI's accuracy, then why would you trust an AI "AI-detector" 🤨

[plutarch@gotosocial.plutar.ch]

@Em0nM4stodon

[em0nm4stodon@infosec.exchange]

@plutarch Hahahahaha amazing

[mmcknett@hachyderm.io]

@plutarch @Em0nM4stodon this explains EVERYTHING

[dalias@hachyderm.io]

@Em0nM4stodon @plutarch Yeah, but I mean this is totally expected and probably what folks asking "is this AI generated" actually mean on a technical level:

"Is this text in the interpolation hull of the corpus?"

What's much more harmful is when it classifies somebody's new genuine writing that's their own truth, and that's not part of the corpus, as "AI generated" because the answer isn't a function of the text itself but of the provenance.

[aud@fire.asta.lgbt]

@Em0nM4stodon@infosec.exchange Those "AI detector" services are worse than useless, anyway, statistically speaking (because they're harmful), but they're also just... not really possible.

An analog would be like trying to determine if your ruler was accurate... by measuring it against itself. It just doesn't work or make any sense.

[dalias@hachyderm.io]

@aud @Em0nM4stodon It's also useless because the answer is simply not a function of the text. The *exact same paragraph* can be either "AI generated" or "not AI generated" and the difference is the provenance - the process by which it came to exist and correspond (or not correspond) to anyone's actual model of reality.

[dalias@hachyderm.io]

@aud @Em0nM4stodon Just like how the exact same paragraph can be "plagiarism" or "not plagiarism" depending on whether the person who wrote it had any knowledge of someone else having written the exact same words. It's all a matter of provenance not text contents.

[falken@social.falkensweb.com]

@aud @Em0nM4stodon see also "halting problem".
Feed tweaked AI slop to AI slop detector till it fails to detect as slop.

#llm

[aud@fire.asta.lgbt]

@jafra@sueden.social @Em0nM4stodon@infosec.exchange

you can measure that they fit and that the negative fits to the ruler.

sure... but that doesn't tell you whether your ruler is accurate. A theoretical "AI detector" that works this way would be trying to determine if two samples come from the same distribution or two different distributions.

My analogy wasn't perfect and was just focused on one of the "quick" flaws of the approach (a biased instrument can't tell whether its biased), but the more fundamental problem (to rephrase what @dalias@hachyderm.io pointed out) is that the distribution inside the LLM is sampled from human created text...

... and so determining that a piece of text "matches" what an LLM might output doesn't tell you whether it was generated by a human or an LLM. It just tells you, at best, whether it could be generated or otherwise found within the internal LLM distribution, and considering they've hoovered up every piece of text they've been able to get their hands on... I mean, I think there's plenty of examples of stuff like "I fed Moby Dick in and it says AI generated". The reason em dashes are outputted are because, well, humans like em dashes... etc etc.

It's just a very bad and wrong approach and is just the tech bros selling the idea of a cure for an unnecessary problem they created.

Even if they use a non LLM approach the whole idea of automated "AI detection" is just an arms race and is likely to be useless before it can even get out the door.

[simplicator@federate.social]

@Em0nM4stodon Guess it doesn’t take one to know one… when the “one” in question can’t actually “know” anything #AI

[sidalsolgun@infosec.exchange]

@Em0nM4stodon my teacher in Canada told me "you're using AI" just because my homework was great. (It wasn't that good imo)

I defended myself but I still wonder what makes him think that, like AI detector or just a guess?

I wouldn't say my homework was great anyways, I did it with my shitty English.

The same thing is happening to other students too like they're doing the homework by themselves but the AI detector says "it's AI".

What a year to live in...

Edit: And it's so fuckin rude and disgusting to say "you're using AI" in the middle of the class. Jfc...

[retreival9096@hachyderm.io]

@sidalsolgun @Em0nM4stodon

If someone said that to my child, I'd ask if the "AI detector" was an LLM -- because this sounds like the teacher accusing the kid of doing exactly what the teacher is doing.

[lhbm@mastodon.social]

@falken @aud @Em0nM4stodon that's exactly what some school kids do

[reformed_content_marketer@mastodon.social]

@Em0nM4stodon There have been plenty of times when an AI "AI-detector" claimed my completely human-written content was partially AI-generated. These tools are so stupid, yet so many people trust them.

I wish there were a reliable, non-privacy-invasive way of assuring human provenance so we could stop these kinds of false accusations. I want there to be reliably AI-free spaces again. All I can really do is use the trust system and hope that showing someone a doc's history occasionally is enough

[meejah@mastodon.social]

@Retreival9096 @sidalsolgun @Em0nM4stodon You might not know if a teacher said this to your child...

It's terrible and stupid and gross, and a very large school board where I live won't even intelligently talk about this stuff (Calgary Board of Education)

[epic_null@infosec.exchange]

@reformed_content_marketer @Em0nM4stodon Okay so this thought has only been in the oven like 5 minutes so forgive me but...

It seems to me this is an extension of our ongoing society-wide trust crisis.

We don't trust the news (why would we at this point)
We don't trust the government (for obvious reasons)
We don't trust each other
We don't trust AI (we shouldn't)

which kinda ties to our accountability crisis

and it feels like there's a final place to go from here but this idea is gonna need a lot time before it's ready.

[sidalsolgun@infosec.exchange]

@meejah @Retreival9096 @Em0nM4stodon

TDSB and others shouldn't be any different I guess.

Québec may be different as their education system is different AFAIK.

[reformed_content_marketer@mastodon.social]

@Epic_Null @Em0nM4stodon yeah, 100% agree. I don't even trust myself anymore half the time. I don't think I'm going to like whatever that final place to go from here is. Still thinking about how to determine what's trustworthy these days.

[retreival9096@hachyderm.io]

@meejah @sidalsolgun @Em0nM4stodon
I am fortunate my children are grown. Things weren't smooth for them, but this is one thing they didn't have to contend with.

[bcasiello@floss.social]

@Em0nM4stodon
AI vendors are probably highly motivated to make AIs lean toward false positives.
1. The more people see AI-identified stuff, the more ubiquitous and inevitable it seems.
2. If it's any good, AI gets the credit.

If they could actually detect quality, you can bet that only crap would get identified as 'not AI'.