When AI Helps Students Learn—and When It Makes Learning Worse

School Leadership 2.0 | Summary for Educators | November 12, 2026

Based on: “Why AI Tools Work Differently on Middle Schoolers Compared to College Students, According to a Researcher”
Author: Jill Barshay, The Hechinger Report, published in KQED MindShift, November 2, 2026
Original Article: Read the full article

THE BIG IDEA

Artificial intelligence holds enormous promise for education, but new research suggests that the way AI is introduced into classrooms may matter more than the technology itself.

Two recent studies led by Jing Liu, director of the Center for Educational Data Science and Innovation at the University of Maryland, produced strikingly different results.

In one study, an AI-powered mathematics tutor was associated with improved achievement among middle school students. In another, introducing an AI assistant into university courses actually coincided with lower grades and reduced student engagement.

The findings challenge the assumption that giving students access to sophisticated AI tools automatically improves learning.

For K–12 educators, the message is particularly important: AI should strengthen students' thinking, encourage meaningful practice, and support teacher-directed instruction—not replace the intellectual effort essential to learning.

The real question for school leaders is no longer simply whether to introduce AI, but how to ensure that AI produces deeper learning rather than greater dependence.

KEY TAKEAWAYS

1. Middle School Students Benefited From AI-Supported Math Practice

During the 2025–26 school year, approximately 6,000 middle school students in Arizona had access to an AI mathematics platform called Goblins.

Students who completed at least two problems per week—more than 50 over the school year—scored significantly higher on standardized mathematics assessments.

The difference was approximately 0.11 standard deviations, estimated as equivalent to three additional months of learning.

However, researchers emphasized that the study was observational. It did not establish that the AI platform caused the gains.

2. Engagement May Be AI's Greatest Opportunity

One particularly promising finding involved student participation.

Half of the participating middle school students used the platform for at least two problems per week, an encouraging result given the persistent difficulty that educational software developers face in sustaining student engagement.

The platform allowed students to communicate with an AI tutor through spoken language, text, and visual demonstrations.

Multilingual capabilities also helped make the technology accessible to English learners.

3. College Students Experienced Unexpected Negative Results

A separate randomized experiment at the University of Maryland involved approximately 2,300 college students.

Some courses incorporated an AI assistant trained on course materials, while comparable courses did not.

Students in AI-supported courses earned grades averaging 0.11 points lower on a four-point scale.

They also participated less frequently in online discussions, interacted less with course materials, and were less likely to seek assistance from professors or classmates.

The negative effects were particularly pronounced among first-generation college students.

4. Easy Access to Answers Can Undermine Learning

Only 15 percent of students in the AI-supported college courses actually used the provided assistant.

Researchers nevertheless suspected that making AI readily available may have encouraged broader reliance on other generative AI tools.

Students might have interpreted the assistant's presence as permission to delegate academic tasks to technology.

This explanation remains a hypothesis rather than a demonstrated cause.

The concern is familiar: When technology completes the thinking, students may complete assignments without developing understanding.

5. Teachers Remain Essential

The middle school program incorporated direct teacher involvement.

Teachers selected problems, set assignments, and customized practice activities.

By contrast, the university assistant primarily served as an always-available resource.

This difference raises an important instructional question: Does AI work best when it supports a teacher-designed learning process rather than functioning as an independent academic shortcut?

6. Schools Must Evaluate Outcomes, Not Just Usage

These studies demonstrate why school systems should resist equating successful technology implementation with widespread adoption.

Administrators must examine what students are actually learning, whether they remain intellectually engaged, and how technology influences their relationships with teachers.

WHY IT MATTERS FOR SCHOOL LEADERS

School districts are facing mounting pressure to adopt AI applications, often before sufficient evidence exists regarding their long-term educational effects.

These findings suggest that age alone does not explain whether AI helps or hurts. Instructional design, student behavior, teacher oversight, and the nature of academic tasks all influence outcomes.

Importantly, the studies do not establish that AI benefits middle schoolers while harming college students generally. They evaluated different tools in different contexts.

For principals and superintendents, the central leadership responsibility is to ensure that technology serves clearly defined instructional purposes rather than to determine those purposes.

FIVE LEADERSHIP ACTION STEPS

1. Pilot before purchasing. Test AI applications with limited groups of teachers and students before committing to districtwide implementation.

2. Require evidence of learning. Examine achievement, student work, participation, and independent problem-solving rather than relying solely on usage statistics.

3. Keep teachers in control. Prioritize applications that enable teachers to assign tasks, monitor student thinking, and intervene when necessary.

4. Teach responsible AI use. Help students distinguish between using AI for explanations, feedback, and practice and using it to avoid the thinking required for learning.

5. Monitor equity and engagement. Investigate whether AI benefits all learners, including students with disabilities, English learners, and students who need additional academic support.

LEADER REFLECTION

If your school introduced an AI tool tomorrow, how would you determine whether students were actually learning more—or simply completing their work more easily?

The most successful schools will not necessarily be those that adopt AI the fastest. They may be the ones who ask the most thoughtful questions about its instructional value.