Clark School Home UMD

ISR News Story

Carol Espy-Wilson is PI for multi-site NSF speech recognition grant

Professor Carol Espy-Wilson (ECE/ISR) is the principal investigator for a two-year, $600,000 National Science Foundation Collaborative Research award, “Multilingual Gestural Models For Robust Language-Independent Speech Recognition.”

This multi-site grant includes researchers at the Stanford Research Institute (SRI), Boston University and Haskins Laboratories. Espy-Wilson’s former student Vikramjit Mitra (EE Ph.D. 2011) is the principal investigator on the portion of the grant going to SRI.

The researchers will develop a large-vocabulary speech recognition system based on articulatory information.

Current state-of-the-art automatic speech recognition (ASR) systems typically model speech as a string of acoustically-defined phones and use contextualized phone units, such as tri-phones or quin-phones to model contextual influences due to coarticulation. Such acoustic models may suffer from data sparsity and may fail to capture coarticulation appropriately because the span of a tri- or quin-phone's contextual influence is not flexible. In a small vocabulary context, however, research has shown that ASR systems which estimate articulatory gestures from the acoustics and incorporate these gestures in the ASR process can better model coarticulation and are more robust to noise.

The researchers will investigate the use of estimated articulatory gestures in large vocabulary automatic speech recognition. Gestural representations of the speech signal are initially created from the acoustic waveform using the Task Dynamic model of speech production. These data are then used to train automatic models for articulatory gesture recognition where the articulatory gestures serve as subword units in the gesture-based ASR system. The research will evaluate the performance of a large-vocabulary gesture-based ASR system using American English (AE). This system will be compared to a set of competitive state-of-the-art recognition systems in term of word and phone recognition accuracies, both under clean and noisy acoustic background conditions.

The broad impact of this research is threefold: (1) the creation of a large vocabulary AE speech database containing acoustic waveforms and their articulatory representations, (2) the introduction of novel machine learning techniques to model articulatory representations from acoustic waveforms, and (3) the development of a large vocabulary ASR system that uses articulatory representation as subword units.

The robust and accurate ASR system for AE will deal effectively with speech variability, thereby significantly enhancing communication and collaboration between people and machines in AE, and with the promise to generalize the method to multiple languages. The knowledge gained and the systems developed will contribute to the broad application of articulatory features in speech processing, and will have the potential to transform the fields of ASR, speech-mediated person-machine interaction, and automatic translation among languages.

This work will result in a set of databases and tools that will be disseminated to serve the research and education community at large.

Espy-Wilson is a University of Maryland 2012 Distinguished Scholar-Teacher. She will give a lecture to the university community, “Say What? Production, Perception and Variability of Speech” on Dec. 7.

Related Articles:
Vishnubhotla, Espy-Wilson granted patent for improving speech extraction
Espy-Wilson named International Speech Communication Association Fellow
Espy-Wilson's technology included in new Alcatel MOVE TIME smart watch
Espy-Wilson named to NIH advisory council
Espy-Wilson delivers plenary address at College Board conference
Engineering systems for mental health work by Espy-Wilson, Resnik, Vaughn-Cooke featured in Newsweek
Depireux, Elhilali co-edit auditory cortex techniques handbook
OmniSpeech, Espy-Wilson mentioned on "Washington Business Report"
Espy-Wilson, Bergbreiter receive ADVANCE Seed Grants
Espy-Wilson gives keynote address at University of Michigan

September 28, 2012


Prev   Next

 

 

Current Headlines

Nau, Gelfand, Goldstein part of MURI developing the potential of mean-field game theory

ECE Names 2017-2018 Distinguished Dissertation Fellows

UMD Celebrates Invention of Year Winners, Ventures, and Partnerships at 'Innovate Maryland'

Ulukus named Anthony Ephremides Professor in Information Sciences and Systems

Maryland researchers develop computational approach to understanding brain dynamics

Alumnus Raef Bassily joins Ohio State as tenure-track faculty

Maryland researchers awarded $1M DARPA Lagrange program cooperative agreement

Bill Regli featured at Global Industrial Cooperation Conference

Brin Family Prize to Support Excellence in Drone-Related Activity

Schizophrenia drug monitoring device research featured on IEEE Sensors Letters cover

News Resources

Return to Newsroom

Search News

Archived News

Events Resources

Events Calendar