Study Evaluates AI Tutoring Impact in Tennessee Middle Schools

A two-year cluster randomized trial conducted in 18 middle schools in Tennessee assessed the impact of the AI tutor Khanmigo, integrated with Khan Academy, on student math achievement. The trial involved randomly assigned students who used the AI tutor during daily remedial mathematics sessions. Results indicated that the use of Khanmigo raised math achievement by 1.3 national percentile ranks per term, equating to approximately 0.06 to 0.08 standard deviations over a school year. The cumulative effect of a full year of participation was estimated at 0.14 standard deviations. These improvements were similar to those observed with Khan Academy practice without AI assistance. However, engagement with the AI tutor was limited; while 96 percent of students used Khanmigo at least once, the median student interacted with it on only a third of the days they practiced, and in just 17 percent of sessions where they made mistakes. The findings suggest that increasing student engagement with the AI tutor is crucial for realizing its potential benefits in education.