Sanjeev Khudanpur
Department of Electrical and Computer Engineering
Department of computer Science
Center for Language and Speech Processing
The Johns Hopkins University
Discovering the Language of Surgery: Automatic Gesture Induction for Manipulative
Tasks
We describe a framework for modeling and recognition of gestures used in manipulative tasks such as robot assisted surgery. ?The key ingredient of our framework is a hidden Markov model (HMM) of the kinematic signal based on which the recognition must be performed: with the states of the HMM corresponding to gestures or sub-gestures, recognition reduces to a standard inference problem. ?The topology and transition probabilities of the HMM capture gesture dynamics and the compositional structure of the task being performed, while the emission probabilities of the HMM capture the stochastic variability between different realizations of the same gesture.
Two important design considerations in using HMMs for gesture recognition are addressed in this talk: how to automatically learn the inventory of gestures or sub-gestures needed to model the manipulative task, and how to select kinematic features that carry the most information for discriminating between gestures. ?A modified procedure for successive refinement of HMM topology is developed to address the former, while an iterative application of heteroscedastic LDA is found to be quite successful for the latter.
HMMs estimated using these techniques are used to recognize suturing trials by a number of surgeons with different levels of expertise using da Vinci surgical robot. ?Gesture recognition accuracies over 80%, the ability to automatically discover key gestures and subgestures, and the ability to automatically align trials of two different surgeons for comparison are demonstrated.