A Network Connecting School Leaders From Around The Globe
Based on research by André Barcaui | Social Sciences & Humanities Open, Volume 12 | 2025
A randomized controlled study offers an important warning for schools rushing to integrate generative AI into everyday learning. Undergraduate students who studied with unrestricted access to ChatGPT remembered significantly less 45 days later than students who learned through traditional, non-AI methods. The finding does not suggest that schools should ban AI. It suggests something more useful: AI should be designed into learning so that it supports thinking rather than replacing it.
One of education's oldest principles may become even more important in the age of artificial intelligence: learning requires intellectual effort.
In André Barcaui's randomized controlled trial, 120 undergraduate students studying artificial intelligence concepts were assigned either to use ChatGPT as a study aid or to rely on traditional study methods. Forty-five days later, students were unexpectedly tested on what they remembered.
The difference was substantial. Students who studied traditionally averaged 68.5%, while students who used ChatGPT averaged only 57.5%—an approximately 11-percentage-point gap. The effect size was moderately large, suggesting that the difference was educationally meaningful rather than merely statistical.
The likely explanation is cognitive offloading. When technology performs too much of the intellectual work—generating explanations, organizing information, summarizing ideas, or producing answers—students may complete tasks successfully without engaging deeply enough to build durable knowledge.
The danger, therefore, may not be that AI gives students wrong answers. It may be that AI gives them good answers too easily.
Students using ChatGPT retained less knowledge. After 45 days, the AI-assisted students averaged 57.5% compared with 68.5% for traditional learners.
AI users also spent considerably less time studying. They averaged about 3.2 hours, compared with 5.8 hours for traditional learners—a reduction of roughly 45%.
Study time did not explain the entire difference. Even after researchers statistically controlled for time-on-task, students in the traditional group retained more information.
The greatest disadvantage appeared with technical material. This raises particular questions for mathematics, science, programming, and other subjects in which students need strong foundational knowledge before tackling increasingly complex material.
Experience with AI did not eliminate the problem. Students who were more familiar with AI were not clearly protected from the retention effect.
The study examined unrestricted AI use—not carefully designed AI-supported instruction. That distinction is crucial. The findings argue for thoughtful sequencing and instructional design, not simply prohibition.
Schools have understandably focused much of the AI conversation on cheating, plagiarism, detection, and academic integrity.
This research suggests a deeper question:
What happens when students can complete academic work without performing the cognitive work that produces learning?
Memory is not an outdated educational objective. Students need stored knowledge in order to recognize patterns, make connections, solve unfamiliar problems, evaluate arguments, and think critically. A student cannot continually outsource every piece of background knowledge required for reasoning.
The study also reinforces the concept of “desirable difficulties.” Retrieval practice, struggling with a problem, generating an explanation, revising an answer, and attempting something before receiving help can feel inefficient. Yet those very difficulties strengthen learning.
Generative AI can remove those difficulties almost instantly.
Schools therefore face an instructional-design challenge: preserving productive struggle while still teaching students how to use one of the most consequential technologies they will encounter.
The researchers themselves caution against overgeneralizing. Participants were university business students at one Brazilian institution, and only 85 of the original 120 students completed the delayed test. The study measured retention rather than every possible benefit of AI, such as creativity, efficiency, synthesis, collaboration, or problem solving.
It is one important study—not a final verdict on AI.
1. Adopt an “AI after attempt” principle.
Require students to wrestle with a question, problem, text, or writing task before turning to AI.
2. Protect AI-free learning experiences.
Some reading, writing, discussion, problem solving, and assessment should deliberately occur without generative AI.
3. Use AI as a tutor rather than an answer machine.
Ask AI to provide hints, questions, feedback, counterarguments, examples, or explanations instead of finished answers.
4. Require retrieval without AI.
After students use AI to explore material, have them close the tool and explain, write, discuss, or solve problems from memory.
5. Assess the learning—not merely the product.
Oral explanations, conferences, drafts, demonstrations, handwritten responses, and in-class problem solving can reveal whether students actually understand what they submit.
6. Teach students about cognitive offloading.
AI literacy should include understanding when technology helps thinking and when it quietly replaces it.
The central question for school leaders may no longer be:
“Are our students using AI?”
A much better question is:
“When students use AI, are they still doing enough of the thinking to learn?”
The goal should not be to create AI-free schools or AI-dependent schools.
It should be to create schools in which students know when to think without AI, when to think with AI, and how to recognize the difference.
This study is particularly useful for your continuing focus on productive struggle and the risks of AI replacing rather than extending student thinking. The next image could work especially well with a visual contrast between a student struggling productively and another allowing AI to carry the cognitive load. DOI
------------------------------
Prepared with the assistance of AI software OpenAI. (2026). ChatGPT (5.2) [Large language model]. https://chat.openai.com
Tags:
SUBSCRIBE TO
SCHOOL LEADERSHIP 2.0
Feedspot named School Leadership 2.0 one of the "Top 25 Educational Leadership Blogs"
"School Leadership 2.0 is the premier virtual learning community for school leaders from around the globe."
---------------------------
Our community is a subscription-based paid service ($19.95/year or only $1.99 per month for a trial membership) that will provide school leaders with outstanding resources. Learn more about membership to this service by clicking one of our links below.
Click HERE to subscribe as an individual.
Click HERE to learn about group membership (i.e., association, leadership teams)
__________________
CREATE AN EMPLOYER PROFILE AND GET JOB ALERTS AT
SCHOOLLEADERSHIPJOBS.COM
Mentors.net - a Professional Development Resource
Mentors.net was founded in 1995 as a professional development resource for school administrators leading new teacher induction programs. It soon evolved into a destination where both new and student teachers could reflect on their teaching experiences. Now, nearly thirty years later, Mentors.net has taken on a new direction—serving as a platform for beginning teachers, preservice educators, and
other professionals to share their insights and experiences from the early years of teaching, with a focus on integrating artificial intelligence. We invite you to contribute by sharing your experiences in the form of a journal article, story, reflection, or timely tips, especially on how you incorporate AI into your teaching
practice. Submissions may range from a 500-word personal reflection to a 2,000-word article with formal citations. Click here to explore and contribute an article.
Also, consider the exciting learning opportunity offered through the Mentors.net AI Academy.