Markerless tongue pose estimation in cross-linguistic spoken and mimed speech using ultrasound

Abstract

Studies of articulatory variation in mimed speech, a proxy for alaryngeal speech, have been restricted to monolingual speakers. This study presents a single-subject case study that investigates cross-linguistic phonological influence of vowels in a 25-year-old female Malayalam-English bilingual speaker, producing spoken and mimed speech. Ultrasound tongue imaging is used to acquire articulatory data, and novel techniques for data analysis are used including pose estimation with DeepLabCut and geometric morphometric analysis (GMM) for quantitative measurement of tongue variation. The results reveal significant differences in tongue kinematics across languages and conditions, without direct indication of phonological transfer. However, within-speaker articulatory variation is evident across the independent variables. These findings will aid in the development of silent speech interfaces for voice restoration in bilingual laryngectomy patients.