A comparative study of learning effects from human-made and AI-generated teaching videos

By Feifei Wang, The Centre for Information Technology in Education, The University of Hong Kong

In the age of generative artificial intelligence (AI), can generative AI-made teaching videos deliver learning outcomes comparable to those of human-made videos? A recent study explored this question by comparing the effectiveness of four generative AI-made videos with four human-made videos on the same topics in a management course. The online intervention involved  447 US-based laypeople, most of whom had completed higher education. The study included two experimental conditions:  in the first, 213 participants watched four teaching videos in a human-AI-human-AI sequence; in the second, 234 participants watched them  in an AI-human-AI-human sequence. After each video, participants completed a survey rating their learning experience, followed by a  multiple-choice exam to assess learning outcomes.

Results showed that human-made videos provided a small but statistically significant advantage in learning experience, suggesting that the participants still preferred human teachers in the videos. Regarding learning outcomes, participants in both conditions achieved a similar degree of acquired knowledge. The authors noted that as generative AI-made teaching videos become increasingly common due to their ease of production, researchers and educators should critically explore ways to use generative AI-made content to improve learning experiences, while being mindful of potential limitations and ethical considerations in production and use.