Students Who Communicate Better With AI Learn Faster. Here's What Our Data Shows
StudyFetch's Learn Engine tracks how students master topics across the platform. It measures performance through an ELO-like scoring system: a mastery score that goes up when students answer correctly and down when they don't, calibrated against question difficulty and other learners' performance. It updates continuously across every interaction, from flashcards to practice exams to AI tutor sessions.
We looked at how students' AI literacy grades correlate with their mastery performance. The results are consistent: students who communicate better with AI and use it more responsibly perform better on assessments, engage more with practice, and master topics at higher rates.
We can't yet prove that better AI communication causes better learning. But the association is strong enough that it's worth understanding, and worth studying further.
What the Learn Engine Is
The Learn Engine's job is to figure out the best content to teach a given topic, and deliver it to the right student at the right time.
StudyFetch currently tracks 30,000 active topic clusters, part of over 100 million total topics and growing. A topic is a concept a student needs to learn. A student taking economics needs to understand "supply and demand." Every time a user interacts with that topic, whether by flipping a flashcard, asking the AI tutor a question, or taking a practice quiz, that interaction feeds the Learn Engine.
Because StudyFetch is AI-native, all content on the platform is generated by AI: flashcards, practice exams, video lessons, podcasts, tutor sessions, scenarios. Because language models are not perfectly deterministic, this naturally creates variation in the content delivered to students. Some content leads to a student answering a quiz question correctly. Some doesn't. The system is optimized to find the path that gets each learner to mastery the fastest.
The system learns at three levels. At the personal level, it identifies what type of content helps a specific student learn fastest, which often comes down to what keeps them engaged enough to keep studying. At the local level, it finds patterns within a particular class or school. At the global level, it looks across all students who studied a topic and finds what helped them master it fastest.
The key difference from other AI platforms: the Learn Engine doesn't optimize for the machine completing the work. It optimizes for the human being able to do the work.
The Study
We sampled 200 students per AI literacy grade (A, B, C, D) across both prompting and responsibility dimensions, ran the analysis 10 times, and averaged the results. We excluded topics that were created but never interacted with. We measured each student's correct rate, mastery rate, average number of questions answered, K-score (our ELO-like mastery metric), and average streak length.
Prompting Grade vs Performance
Grade Students Avg K-Score Correct Rate Mastery Rate Avg Q's Avg Streak
A 200 1600.9 71.1% 28.0%. 150.0 3.56
B. 200 1588.5 69.1% 20.7% 211.0 2.99
C 200 1579.1 65.3% 19.9% 174.2. 2.68
D 200 1569.7 64.5% 18.8% 75.6 2.61
Students with A-grade prompting scores answer 71.1% of questions correctly vs 64.5% for D-grade students. K-scores decline from A to D without exception. A-grade students master topics at 28.0% vs 18.8% for D-grade students, a 1.5x difference.
Responsibility Grade vs Performance
Grade Students Avg K-Score Correct Rate Mastery Rate Avg Q's Avg Streak
A 200 1605.3 72.1% 24.9% 158.5 3.31
B 200 1583.4 68.3% 21.3% 154.3 3.06
C 200 1573.5 67.8% 20.6% 109.8 2.86
D 63 1554.8 61.5% 19.8% 50.1 3.08
Responsibility shows an even stronger correlation. The correct rate gap between A and D students is 10.6 percentage points (72.1% vs 61.5%), nearly double the prompting gap. K-scores again decline monotonically from A to D.
Note: the D-responsibility bucket contains only 63 students (only 11 with K-score data), so those numbers should be interpreted with caution.
The Engagement Gap
The most striking finding may not be the performance difference but the engagement difference.
D-grade prompting students answer half as many questions as A-grade students: 75.6 vs 150.0. D-grade responsibility students answer just a third as many: 50.1 vs 158.5.
Students who struggle to communicate with AI don't just perform worse. They practice less. Whether this is because poor AI interactions are discouraging, or because lower-engagement students also happen to be worse prompters, or both, the result is the same: the students who most need practice are getting the least of it.
This raises a product question we're actively working on. If a student's poor prompting is leading to unhelpful AI responses, and those unhelpful responses are causing the student to disengage, then teaching better prompting isn't just an AI literacy intervention. It's a retention intervention.
Even Controlling for Engagement, the Gap Persists
One possible explanation for the performance gap is simply that A-grade students practice more, and more practice leads to better scores. But the mastery rate data complicates that story.
On topics they actually practiced, A-grade prompting students master 28.0% of topics vs 18.8% for D-grade students. That's a 1.5x difference that can't be explained by volume alone. If practice were the whole story, you'd expect similar mastery rates per topic across grades, with A students just covering more topics. Instead, A students are also more effective per topic.
What This Does and Doesn't Tell Us
This data shows a consistent association between AI literacy and learning performance. It does not prove causation.
The most important alternative explanation: general academic ability. Stronger students may naturally write better prompts, use AI more responsibly, and perform better on assessments, all independently. If that's the case, the AI literacy score is reflecting an underlying trait, not a teachable skill that directly improves outcomes.
We can't rule this out with observational data. To separate the effect of AI literacy from the effect of general ability, we would need a controlled study: take students with similar baseline ability, teach some of them to prompt better, and measure whether their learning outcomes improve relative to a control group.
That study hasn't been done yet. But the data makes a case that it should be.
The correlation is consistent across every metric we measured: correct rate, K-score, mastery rate, engagement volume, streak length. It holds for both prompting and responsibility, with responsibility showing the stronger association. And the engagement gap suggests a potential mechanism: better AI communication may lead to more useful AI responses, which may keep students practicing longer, which compounds into better outcomes.
If that chain holds up under controlled conditions, it means AI literacy instruction isn't just about preparing students for the workforce. It's about helping them learn better right now.
What's Next
We're pursuing partnerships with researchers and institutions to run controlled studies on this question. If better prompting does cause better learning outcomes, the implications go beyond StudyFetch. It would mean that teaching students to communicate with AI improves every subsequent AI interaction the student has, across every subject and every tool.
If you're a researcher interested in studying the causal link between AI literacy and learning outcomes, we have the data infrastructure to run that study at scale. We'd welcome the collaboration.
