Human-in-the-Loop AI Item Generation and On-Demand AI Debugging in Programming Education: An Exploratory Nonequivalent-Groups Study

Résumé fourni par la source

This exploratory study examined a dual-role artificial intelligence-supported programming environment that integrated human-in-the-loop item generation and on-demand AI debugging within a flipped cognitive apprenticeship framework. Programming tasks were initially generated by GPT-4o and were subsequently reviewed, revised, and approved by the instructor before release. Both instructional conditions completed the same instructor-approved tasks and received the same compiler and automated test-case feedback. Students in the AI Debug-enabled condition could additionally request diagnostic guidance after an unsuccessful submission, whereas students in the comparison condition did not have access to AI-generated debugging assistance. The analytic sample comprised 45 students in the AI Debug-enabled condition and 16 students in the comparison condition. Prior-semester programming achievement, operationalized as the total grade obtained in the preceding programming course, was used as an archival baseline covariate. The observed baseline difference was not statistically significant. Programming-task performance was recorded across 13 instructional weeks. A generalized estimating equation model controlling for prior-semester programming achievement showed no significant overall condition effect, but revealed significant effects of instructional week and the Condition × Week interaction. Prior-semester programming achievement significantly predicted weekly task performance. The comparison condition also obtained higher midterm and final examination scores after adjustment for prior achievement. The findings do not demonstrate a consistent programming-performance advantage associated with access to AI Debug. Instead, adjusted differences varied across instructional weeks and assessment contexts. Because the study used naturally occurring nonequivalent groups and did not retain AI Debug activation records, the results should be interpreted as exploratory associations concerning feature availability rather than causal effects of actual AI use. The primary contribution lies in documenting a human-supervised architecture that integrates instructor-facing AI item generation and student-facing debugging assistance while distinguishing AI-supported task performance from independent programming achievement.

Ce résumé expose les affirmations des auteurs. BNTIC ne l’interprète pas comme une validation indépendante des résultats.