Abstract

On the computational aspects of sign language recognition. Talk given at the Math & Computer Science Department, Gallaudet University, November 21, 2003.

Full-color handouts (460K) - Black and white handouts (suitable for printing) (350K)

In this talk I will look at sign language recognition from a computational point of view. At a first glance, it seems that sign language recognition is sufficiently similar to speech recognition to make adaptation of speech recognition methods straightforward.

On a closer look, this is not the case. There are many issues specific to sign language recognition that require - in some cases - extensive modifications to the basic speech recognition framework.

In particular, I will discuss how the classical hidden Markov model-based recognition framework needs to be changed to account for the inherent simultaneity in sign languages - that is, to accommodate simultaneous handshape changes and hand movements. In addition, I will discuss how best to represent the sign language data, so as to maximize recognition rates.