Redesigning Automated Scoring Engines to Include Deep Learning Models

Le résumé fourni par la source

This chapter describes the rationale and approach for redesigning an automated scoring system to include deep learning models to score both essays and constructed response items. Basic concepts of deep learning, particularly the transformer, are discussed. The principles underlying the engine re-design are covered, including the architecture design that supports the ability to add new models and to deploy models to meet performance requirements. Issues faced when training and deploying transformer models are described, including how these were mitigated. These issues include length limitations on responses, model overconfidence, need for specialized hardware and software, and the difficulty in interpreting large models. Exemplar performance results comparing the results of the original and new results are also presented and discussed. The chapter concludes with a discussion of future work, including plans for interpretability, feedback, efficiency, and use of generative artificial intelligence.

Ce résumé expose les affirmations des auteurs. BNTIC ne l’interprète pas comme une validation indépendante des résultats.