Automated Scoring of Tablet-Administered Expressive Language Tests

Frontiers in Psychology

article

Robert Gale¹  Julie Bird¹  Yiyi Wang²  Jan van Santen¹  Emily Prud'hommeaux²  Jill Dolata¹  Meysam Asgari¹ 

[1] Center for Spoken Language Understanding, Oregon Health & Science University (OHSU), United States;Boston College, United States

关键词: speech;    expressive language;    language disorders;    assessment;    automated scoring;    neural language models;   

DOI  :  10.3389/fpsyg.2021.668401

学科分类：社会科学、人文和艺术（综合）

来源: Frontiers

【 摘 要 】

Speech and language impairments are common pediatric conditions, with as many as 10% of children experiencing one or both at some point during development. Expressive language disorders in particular often go undiagnosed, underscoring the immediate need for assessments of expressive language that can be administered and scored reliably and objectively. In this paper, we present a set of highly accurate computational models for automatically scoring several common expressive language tasks. In our assessment framework, instructions and stimuli are presented to the child on a tablet computer, which records the child's responses in real time, while a clinician controls the pace and presentation of the tasks using a second tablet. The recorded responses for four distinct expressive language tasks (expressive vocabulary, word structure, recalling sentences, and formulated sentences) are then scored using traditional paper-and-pencil scoring and using machine learning methods relying on a deep neural network-based language representation model. All four tasks can be scored automatically from both clean and verbatim speech transcripts with very high accuracy at the item level (83−99%). In addition, these automated scores correlate strongly and significantly (ρ = 0.76–0.99, p < 0.001) with manual item-level, raw, and scaled scores. These results point to the utility and potential of automated computationally-driven methods of both administering and scoring expressive language tasks for pediatric developmental language evaluation.

【 授权许可】

CC BY