Using #AI makes people less likely to be correct
-
Using #AI makes people less likely to be correct
Accuracy collapsed when AI was available
- Baseline: 27% gave correct answer
- With AI advice, 9% gave #correct answerFact: Some would-be correct people got AI advice & became #wrong
AI advice made people #confident there're correct
- Baseline: 30%
- With AI help, confidence rose to 76%People became worse, accuracy was only 1/3, but they were twice as confident
#digital #DumbedDown #DDD = Digitally Dumbed Down
@paka @fallenhitokiri Seems like a bad study if only 44% can get it correct with or without LLMs. Especially if you tell them to use a bad model you know will fail on subjects they know nothing about. Paying them incentivized them to care slightly more but clearly not enough to abandon the tool you asked them to use.
-
Using #AI makes people less likely to be correct
Accuracy collapsed when AI was available
- Baseline: 27% gave correct answer
- With AI advice, 9% gave #correct answerFact: Some would-be correct people got AI advice & became #wrong
AI advice made people #confident there're correct
- Baseline: 30%
- With AI help, confidence rose to 76%People became worse, accuracy was only 1/3, but they were twice as confident
#digital #DumbedDown #DDD = Digitally Dumbed Down
@paka AI doesn't allow you any time to chew on the information you get from it, because it is always ready to go with more information for you. There should be time delayed version of it, so that you can sit with the stuff it just said before asking for some more.
When two humans discuss something there is thinking time, moments when neither says anything. AI just keeps offering to take you further into what may or may not be facts. None of it stays permanently in your head.
-
@paka @fallenhitokiri Seems like a bad study if only 44% can get it correct with or without LLMs. Especially if you tell them to use a bad model you know will fail on subjects they know nothing about. Paying them incentivized them to care slightly more but clearly not enough to abandon the tool you asked them to use.
-
@fallenhitokiri I'm all for a good study, but it reads like taking @treyhunner's https://whereabouts.earth/daily/ app, adding an AI mode where you ask people to use the tool, but you give them wrong answers on purpose for your study when you know the tool will mostly give the wrong answer.
I agree on narrative, but I fail to see "research"
-
T tanyakaroli@expressional.social shared this topic
-
Using #AI makes people less likely to be correct
Accuracy collapsed when AI was available
- Baseline: 27% gave correct answer
- With AI advice, 9% gave #correct answerFact: Some would-be correct people got AI advice & became #wrong
AI advice made people #confident there're correct
- Baseline: 30%
- With AI help, confidence rose to 76%People became worse, accuracy was only 1/3, but they were twice as confident
#digital #DumbedDown #DDD = Digitally Dumbed Down
@paka ai is a miracle any way you slice it
-
Using #AI makes people less likely to be correct
Accuracy collapsed when AI was available
- Baseline: 27% gave correct answer
- With AI advice, 9% gave #correct answerFact: Some would-be correct people got AI advice & became #wrong
AI advice made people #confident there're correct
- Baseline: 30%
- With AI help, confidence rose to 76%People became worse, accuracy was only 1/3, but they were twice as confident
#digital #DumbedDown #DDD = Digitally Dumbed Down
@paka Dunning-Kruegerizers, automated and deployed to scale.
-
@fallenhitokiri I'm all for a good study, but it reads like taking @treyhunner's https://whereabouts.earth/daily/ app, adding an AI mode where you ask people to use the tool, but you give them wrong answers on purpose for your study when you know the tool will mostly give the wrong answer.
I agree on narrative, but I fail to see "research"
@webology @treyhunner an AI mode with provided wrong answers would potentially be better at whereabouts.earth than I am
-
@webology @treyhunner an AI mode with provided wrong answers would potentially be better at whereabouts.earth than I am
@fallenhitokiri @treyhunner My Claude was about 5 for every 6 or 7, mostly because of small boundary lines. I'm terrible at it.
-
J jwcph@helvede.net shared this topic
-
Using #AI makes people less likely to be correct
Accuracy collapsed when AI was available
- Baseline: 27% gave correct answer
- With AI advice, 9% gave #correct answerFact: Some would-be correct people got AI advice & became #wrong
AI advice made people #confident there're correct
- Baseline: 30%
- With AI help, confidence rose to 76%People became worse, accuracy was only 1/3, but they were twice as confident
#digital #DumbedDown #DDD = Digitally Dumbed Down
I like this. One could argue that AI is a Dunning-Kruger Club enabler. And as others have pointed out, 'the first rule of Dunning-Kruger Club is that you don't know you are in Dunning-Kruger Club'.
-
Using #AI makes people less likely to be correct
Accuracy collapsed when AI was available
- Baseline: 27% gave correct answer
- With AI advice, 9% gave #correct answerFact: Some would-be correct people got AI advice & became #wrong
AI advice made people #confident there're correct
- Baseline: 30%
- With AI help, confidence rose to 76%People became worse, accuracy was only 1/3, but they were twice as confident
#digital #DumbedDown #DDD = Digitally Dumbed Down
-
@paka @fallenhitokiri Seems like a bad study if only 44% can get it correct with or without LLMs. Especially if you tell them to use a bad model you know will fail on subjects they know nothing about. Paying them incentivized them to care slightly more but clearly not enough to abandon the tool you asked them to use.
@webology @paka @fallenhitokiri
this was my thought as well, seems like the study set subjects up to fail if they used AI. confidence in AI is interesting, but so is confidence in the study. maybe it says more about a subject’s ability to discern AI models than anything else.
-
Using #AI makes people less likely to be correct
Accuracy collapsed when AI was available
- Baseline: 27% gave correct answer
- With AI advice, 9% gave #correct answerFact: Some would-be correct people got AI advice & became #wrong
AI advice made people #confident there're correct
- Baseline: 30%
- With AI help, confidence rose to 76%People became worse, accuracy was only 1/3, but they were twice as confident
#digital #DumbedDown #DDD = Digitally Dumbed Down
@paka "They used Step 3.5 Flash because it was usually wrong..."
Ofc you pick the AI answer when you don't know the answer yourself, when given this option.Why not try this suggestion, guessing.
-
A anderslund@expressional.social shared this topic
