|
Rapid task-related plasticity of spectrotemporal receptive fields in primary auditory cortex |
|
|
Applications and robotic implementations of auditory processing |
|
|
Physiological basis of auditory streaming in auditory cortex |
|
|
Selected Past Projects |
|
|
|
|
|
Rapid task-related plasticity of spectrotemporal receptive fields in primary auditory cortex |
|
|
The view of the cortex as a static processor of sensory information has been giving way to the notion of a malleable system that slowly adapts to a changing environment during development, injury, or stimulation. This view is now evolving again in light of recent studies in primates and ferrets that reveal nearly instantaneous changes that occur in the cortex when animals are engaged in various tasks. For instance, we have recently demonstrated that in an auditory detection task, in which a ferret is trained to detect the presence of a target tone against a background of broadband noise, STRFs undergo rapid facilitation that is often specific to the target frequency, and can be short-lived or alternatively persist for many hours, leading to long-lasting receptive field changes, a form of sensory memory. Furthermore, this rapid plasticity is absent or weak in naïve animals, or in non-behaving and poorly behaving trained animals. Recent results also demonstrate that STRF changes are dependent on the exact nature of the behavioral task and target stimuli involved, but consistent with the goal of optimizing the animal performance under different conditions. The insights provided by these findings, coupled with the powerful tools developed for on-line measurements of the STRFs, provide a strong impetus for pursuing more elaborate experimental designs to explore the underlying mechanisms, computational implications, and optimality principles that govern cortical plasticity. |
|
|
Applications and robotic implementations of auditory processing |
|
|
One of the most exciting aspects of studying auditory function is seeing its tangible benefits in numerous applications ranging from the analysis and efficient coding of speech and music, to the assessment of speech intelligibility, to the development of robust auditory behavior in robots and smart sensors. These applications in turn have provided much inspiration into the nature of auditory perception and have significantly influenced the direction of our experimental research over the years. Several theoretical projects are underway and will likely continue in the next few years: (1) Sonifying images: The objective of this research is to render sound from images by mapping visual attributes to their auditory counterparts. The correspondence between the two modalities is established by abstractions of the underlying neural processing in both systems. (2) Perception of musical timbre: This project seeks to develop a metric for the complex percept of musical timbre based exclusively on the representations of different sounds in models of cortical processing. (3) Speech coding, enhancement, and detection: The aim is to develop a comprehensive auditory-based representation of the speech signal that would explain its remarkable perceptual robustness, and be valuable in the assessment of its intelligibility, its efficient transmission, and its robust automatic recognition. Hardware implementations are also underway motivated by the desire to embed auditory capabilities in miniature robots and smart MEMS microphones. The favorite medium of our projects continues to be the analog VLSI technology, which has proven effective in fabricating extremely low-power neuromorphic structures mimicking cochlear function and sound localization circuits. The primary goal of the next phase is the design and fabrication of multiscale spectrotemporal analyzers resembling the cortical analysis of sound. TOP |
|
|
Understanding the functional organization of the auditory cortex has been the main focus of our experimental research. In pursuit of this goal, several methodologies were developed based on systems-identification concepts to enable rapid and relatively comprehensive measurement of cortical spectro-temporal receptive fields (STRF). Subsequent studies explored the topographic distribution of STRF features in different cortical fields, its ability to predict responses to novel stimuli, and its utility in characterizing nonlinear interactions and precisely-timed firings. The STRF has now become one of the standard descriptors of neuronal selectivity, one that can be employed to characterize processes and phenomena well beyond those conceived of at its inception. For instance, the STRF is now used as an efficient indicator of adaptive change in cortical cell spectral tuning and dynamics induced by behavioral demands, as elaborated above. Furthermore, the insights gained from the detailed analysis of recorded STRFs in AI have led us to the development of detailed computational models of auditory cortical processing which form the backbone of much ongoing algorithmic development and applications. These include the assessment of speech intelligibility, models of auditory scene analysis, and the analysis of various auditory percepts such as timbre, pitch, and localization in the mid-plane. |
|
|
Physiological basis of Auditory Streaming in Auditory Cortex |
|
|
Humans can effortlessly perceive and navigate their acoustic surroundings despite the multiplicity of simultaneous sound sources, and often in noisy and reverberant environments. A critical perceptual phenomenon underlying these remarkable abilities is auditory streaming – the ability to parse different sources into segregated “streams”, and hence to attend to one or another. This innate capability utilizes a wide range of acoustic cues and perceptual attributes (location, pitch, loudness, timbre), and is closely related to other complex phenomena such as the “continuity illusion”. The physiological underpinnings of auditory streaming remain barely explored, hampered by the lack of animal models, by the difficulties of reliably interpreting human imaging data in these tasks, and by the absence of comprehensive computational models of this phenomenon to facilitate the experimental work. Ongoing research in our laboratory aims to integrate these three different strands (computational models, physiology, and psychoacoustics) into a coherent study of auditory streaming. Specifically, we have been developing and testing computational models of cortical function suitable for analysis and validation of data from psychoacoustic data on streaming. Behavioral experiments are also underway to determine whether our animal model (ferret) exhibits reactions consistent with perception of auditory streams. These behavioral paradigms have been adapted for use in pilot physiological experiments to record cortical responses to the same acoustic stimuli. Finally, we have initiated a complementary collaborative project critical to the success of this effort that will combine our animal research with a growing body of human psychoacoustic experiments (conducted at MIT), and with MEG imaging studies of the auditory cortex in humans (conducted at UMD). |
|
|
Encoding the auditory spectrum: lateral inhibition This idea attempted to resolve a fundamental uncertainty in early auditory processing: Is the acoustic spectrum encoded by the pattern of average activation across the tonotopic axis (place code), or by the temporal response patterns it evokes in each auditory-nerve fiber (temporal code)? It explained how specific spatiotemporal response features inherent to cochlear function could encode the acoustic spectrum in the response patterns of the auditory nerve. It also demonstrated how such a code harmoniously combines spatial and temporal cues, how it maintains a faithful spectral representation despite the limited dynamic range of the auditory-nerve, and how it can be readily decoded to extract the acoustic spectrum by plausible lateral inhibitory interactions in the recipient cochlear nucleus. |
|
|
Prevalent models of binaural processing have postulated that sound position is estimated in the binaural nuclei by neural delay-lines and coincidence detectors that compensate for time differences between the two ears. The impetus for the alternative stereausis idea was the challenge posited by the lack of convincing anatomical evidence of delay-lines in mammalian species. The stereausis model proposed that accurate neural delay-lines are unnecessary if binaural coincidence detectors instead combined information from slightly mismatched positions (or frequencies) on the two cochleae, effectively exploiting the accurate “mechanical” delay-line inherent in the cochlear traveling wave. Such a scheme turns out to be analogous to the way stereo-depth perception is derived from spatial disparities. |
|
|
The pitch of complex sounds plays a key role in grouping sound sources and organizing the perception of acoustic environments. Psychoacoustic and physiological studies of pitch have been dominated in recent years by two opposing views. In one, pitch is presumed to be encoded primarily by the periodic temporal response patterns on the auditory nerve, and is extracted by population auto-correlations in midbrain nuclei. The alternative hypothesis is that pitch is derived from the harmonic spectral pattern of the acoustic stimulus by matching it with harmonic templates postulated to exist also in the auditory midbrain. Both theories suffer from a dearth of evidence in support of the biological substrates necessary to perform the computations in the midbrain (e.g., autocorrelations or harmonic templates). The ideas presented in JASA 2001 paper detail how harmonic templates can in fact emerge in the early stages of the auditory pathway as a natural consequence of cochlear function, and what physiological and anatomical substrates one may search for in establishing the presence of such templates in the auditory midbrain. |
|
|
A VLSI cochlea model chip has been developed in the NSL to perform real-time cochlear processing of audio data. While some researchers have used transconductance amplifiers (TCAs) as the building block of their hardware cochlea models, our chip is based upon switched capacitor filters (SCFs). These filters allow for precise frequency response and do not require post-fabrication tuning. During the development of the cochlea model, several advances in the field of SCFs and filter banks were made, and a patent has been granted for this technology. |
|