Using 30 months of panel data on 26,811 Chinese students in grades 7-12, it has been studied how generative AI affects homework productivity and learning.
-
Using 30 months of panel data on 26,811 Chinese students in grades 7-12, it has been studied how generative AI affects homework productivity and learning.
● X-Axis: Homework scores
● Y-Axis: Exam scores@metin this is brutal and also very important. Thanks for sharing!
-
@metin there it is:
@mklovenotcyber @metin wow that is exceptionally bad data visualization.
to me this is really hard to parse, especially since there are two separate "white" shapes only differeniated by border colour
-
@rexyislive Indeed. The reason why I don't use Google as search engine is that it works no more.
@forthy42 @rexyislive I use duck duck go instead of Google and it's much better
-
Using 30 months of panel data on 26,811 Chinese students in grades 7-12, it has been studied how generative AI affects homework productivity and learning.
● X-Axis: Homework scores
● Y-Axis: Exam scoresGood time to not be a lazy idiot I guess, look like a super genius compared to your peers just by doing the basics.
Begs the question though, are people smarter for not using AI or are the smart people the ones already pre-disposed to avoid AI?
-
@metin there it is:
@mklovenotcyber @metin Do I see Generative AI users dropping the absolute best exam scores, the best homework scores, and the lowest homework times?
Seems like a win-win-win!
I guess I gotta read the paper?
-
@forthy42 @rexyislive @metin I sometimes use it as a tutor, a kind of a teacher. A means to self-paced learning on steroids if you will
I never blindly copy the output. I never let it fully complete a task. I let it give me a nudge into a direction that I define. And take it from there.
But that requires a lot of reflection, self-awareness and understanding of the underlying technology, bc giving a command "do X for me" seems so much more convenient
@Ketakater I used to do just that, in the beginning, but if you stop to think about the fact that it had to get the information from somewhere. We never know who really spent the time to write tutorials and explanations, or give them any sort of credit by default.
And a full chat session for learning just takes real world resources, way more than sitting and researching or googl8ng used to.
-
@forthy42 @rexyislive @metin I sometimes use it as a tutor, a kind of a teacher. A means to self-paced learning on steroids if you will
I never blindly copy the output. I never let it fully complete a task. I let it give me a nudge into a direction that I define. And take it from there.
But that requires a lot of reflection, self-awareness and understanding of the underlying technology, bc giving a command "do X for me" seems so much more convenient
@Ketakater @forthy42 @rexyislive @metin
Even at that, LLMs will subtly alter the direction of the nudge — towards median, but also towards various biases -
@mklovenotcyber @metin Do I see Generative AI users dropping the absolute best exam scores, the best homework scores, and the lowest homework times?
Seems like a win-win-win!
I guess I gotta read the paper?
@mklovenotcyber @metin I now really want to read the paper. I really want to know if they did anything beyond looking at mean shifts.
Did they, for example, look at where individuals ranked in the distributions and track where those individuals landed in the distribution after the adoption of AI.
Did the performers of the curve stay at the top?
Did any high performing individuals move up higher on tests?
Did any low performers suddenly shine on tests? -
@Ketakater I used to do just that, in the beginning, but if you stop to think about the fact that it had to get the information from somewhere. We never know who really spent the time to write tutorials and explanations, or give them any sort of credit by default.
And a full chat session for learning just takes real world resources, way more than sitting and researching or googl8ng used to.
@rexyislive also true of course
-
@Ketakater @forthy42 @rexyislive @metin
Even at that, LLMs will subtly alter the direction of the nudge — towards median, but also towards various biases@sabik @forthy42 @rexyislive @metin I know
-
Using 30 months of panel data on 26,811 Chinese students in grades 7-12, it has been studied how generative AI affects homework productivity and learning.
● X-Axis: Homework scores
● Y-Axis: Exam scores@metin This graph is interesting and mostly matches my expectation. If you spend the time to actually do the homework, you perform ok the exam (on average). No matter if you use AI or not.
-
Using 30 months of panel data on 26,811 Chinese students in grades 7-12, it has been studied how generative AI affects homework productivity and learning.
● X-Axis: Homework scores
● Y-Axis: Exam scores@metin I suppose this is one way of shoving bots toward 'human-level' performance...
-
@mklovenotcyber @metin Do I see Generative AI users dropping the absolute best exam scores, the best homework scores, and the lowest homework times?
Seems like a win-win-win!
I guess I gotta read the paper?
@mklovenotcyber@infosec.exchange @poleguy@mastodon.social @metin@graphics.social
Your interpretation of the exam-score graph is consistent with the assumptions I made of you upon seeing that you're pro-GenAI.
-
@metin @PapyrusBrigade I can't understand the graph... What are the axis?
-
@aoanla @mklovenotcyber @metin
Yes, the Economist points this out, too:
“The drop in exam scores was concentrated among students who rushed their homework. Those who used AI but spent as long on assignments as non-users paid little penalty. What matters, then, is how pupils use the technology. Those whose exam results remained strong were not simply copying and pasting answers to save time. More likely they used the chatbots as a personal tutor, perhaps to explain difficult concepts or help solve specific problems.”
https://www.economist.com/graphic-detail/2026/08/18/does-ai-stop-children-from-learning ($)
@gisiger @aoanla @mklovenotcyber @metin But put another way, AI had no gain when being used to study if they spent the same amount of time for similar test scores
-
@mklovenotcyber@infosec.exchange @poleguy@mastodon.social @metin@graphics.social
Your interpretation of the exam-score graph is consistent with the assumptions I made of you upon seeing that you're pro-GenAI.
@dzamie @mklovenotcyber @metin
Gen AI is utilizing the unceded words, thoughts, and images of a multitude... it's burning the planet... I wish it would go away.
The "win-win-win" is a joke. The ":-)" emoji was not enough.
It was not an "interpretation" of the exam-score... just an observation of a ridiculous, yet 'valid' way of reading the graphs... for humorous effect. Sorry if it wasn't clear I was joking.
I'm not sure what posts you saw of mine that caused you to conclude I'm pro-GenAI.
-
Using 30 months of panel data on 26,811 Chinese students in grades 7-12, it has been studied how generative AI affects homework productivity and learning.
● X-Axis: Homework scores
● Y-Axis: Exam scores@metin
Ouch! -
@dzamie @mklovenotcyber @metin
Gen AI is utilizing the unceded words, thoughts, and images of a multitude... it's burning the planet... I wish it would go away.
The "win-win-win" is a joke. The ":-)" emoji was not enough.
It was not an "interpretation" of the exam-score... just an observation of a ridiculous, yet 'valid' way of reading the graphs... for humorous effect. Sorry if it wasn't clear I was joking.
I'm not sure what posts you saw of mine that caused you to conclude I'm pro-GenAI.
@mklovenotcyber@infosec.exchange @poleguy@mastodon.social @metin@graphics.social
You did a good job of emulating the writing style of an AI bro: exuberant, confidently incorrect, and using more smilies than normal (though in hindsight, it being an emoticon rather than an emoji should have given me pause). The tone and "win-win-win" especially gave me my initial assumption, and nothing else about it gave me good reason to reject it.
So, I'm not sure what you mean by "valid," then - I thought at first that it might be a thin red tail above all the blue scores, but I only see the black axis (well, grey guideline) past the blue when I zoom in. I can't think of anything else it could be, though, unless you're trying to suggest spread as a useful metric.
-
Using 30 months of panel data on 26,811 Chinese students in grades 7-12, it has been studied how generative AI affects homework productivity and learning.
● X-Axis: Homework scores
● Y-Axis: Exam scores@metin This reminds me of something a very experienced and smart person told me about aviation safety standards. He said it doesn't matter what's in the standard as long as it makes you, and others, sit with the design longer. He thinks quality is more a function of time than of specific activities.
He'd probably like this paper a lot.
-
@mklovenotcyber@infosec.exchange @poleguy@mastodon.social @metin@graphics.social
You did a good job of emulating the writing style of an AI bro: exuberant, confidently incorrect, and using more smilies than normal (though in hindsight, it being an emoticon rather than an emoji should have given me pause). The tone and "win-win-win" especially gave me my initial assumption, and nothing else about it gave me good reason to reject it.
So, I'm not sure what you mean by "valid," then - I thought at first that it might be a thin red tail above all the blue scores, but I only see the black axis (well, grey guideline) past the blue when I zoom in. I can't think of anything else it could be, though, unless you're trying to suggest spread as a useful metric.
@dzamie @mklovenotcyber @metin Yeah, sometimes jokes don't land when done in text... I tried. ... and I habitually don't use actual emoji's... sticking to the emoticons is a stylistic choice.
By 'valid' I mean that the three things I claimed are factual. I see red at the far right of the "Exams Scores" graph, red at the top of thehomework scores, and red at the _bottom_ of homework time; for homework time _lower_ is better.
As usual, jokes just get funnier when you explain them.
Oh well
