UAHCI'05 Abstract
Christian Vogler and Siome Goldenstein. Analysis of Facial Expressions in American Sign Language. Proceedings of the 3rd Intl. Conf. on Universal Access in Human-Computer Interaction (UAHCI) 2005.
In the age of speech and voice recognition technologies, sign language recognition is an essential part of ensuring equal access for deaf people. To date, sign language recognition research has mostly ignored facial expressions that arise as part of a natural sign language discourse, even though they carry important grammatical and prosodic information. One reason is that tracking the motion and dynamics of expressions in human faces from video is a hard task, especially with the high number of occlusions from the signers' hands. In this paper, we present a 3D deformable model tracking system to address this problem. We apply it to sequences of native signers, taken from the National Center of Sign Language and Gesture Resources (NCSLGR), with a special emphasis on outlier rejection methods to handle occlusions. Our experiments validate the output of the face tracker against expert human annotations of the NCSLGR corpus, demonstrate the promise of our proposed face tracking framework for sign language data, and reveal that the tracking framework picks up properties that ideally complement human annotations for linguistic research.