Can the brain be thought of as a kind of computer? While this may be a topic of debate, few would deny that the brain does perform computations. The subject of my research program is to identify, and describe quantitatively, such neural computations—specifically those performed in the brain’s auditory system. This program both sheds light on the function of the brain, and permits us to discover algorithms (specific steps used in the computations) otherwise unknown to engineering.
The range of problems solved by neural computations span the low-level (e.g. determining the spatial location of a sound source based on the different acoustic signals received by each ear) to the high-level (e.g. in a crowded room, detecting the arrival of a new voice, or the departure of an old one). Often these neural computations are critical to the proper functioning (or survival) of an animal, and so must be performed reliably and quickly, even under adverse circumstances.
These neural computations employ algorithms developed and fine-tuned by millions of years of evolution. As such the computations are typically far beyond the capability of even the most advanced computers. By identifying, understanding, and quantitatively describing the computations performed by the brain, it is possible to determine the algorithms. Understanding the algorithms would have great potential benefits to engineering applications (e.g. auditory-based identification algorithms, robust speaker identification, robust speech processing).
The class of neural computations that use the temporal character of the sounds being processed—those for which time plays an important role—are the primary focus of my research. As will be described below, this class includes the computational roles in the processing of interaural (“between ear”) time differences and correlations, overall modulations of complex sounds, and temporal symmetries used in neural computations.
My research program has components at three different hierarchical levels: auditory neural computations observable macroscopically (at the whole brain level), auditory neural computations at the level of small networks of neurons (e.g. a few dozen neurons involved in a single computation), and auditory neural computations at the level of individual neurons. A fourth research area develops new ideas in the signal processing of neural data and in computational neuroscience.
Auditory processing is observable at the level of the whole brain via a variety of tools. Some fall into the broad category of brain-imaging, such as fMRI, but are typically limited in their access to temporal computations. To use an analogy from photography, these tools have shutter-speeds too slow to see quickly changing temporal aspects of the computations. An alternative without this drawback is magnetoencephalography (MEG), which is sensitive to neural processes changing as fast as every millisecond. MEG is related to the more commonly used clinical tool electroencephalography (EEG), but it has key advantages due to its use of neural magnetic fields instead of electrical fields: the brain is magnetically, but not electrically, transparent.
Using MEG, I can reveal the brain’s strategies for decoding acoustic inputs, and encoding their auditory information, depending on the sound features. I highlight five specific accomplishments:
1) Demonstrating the simultaneous neural encoding of multiple, independent, modulations (ways in which sounds can change in time), with direct relevance to how speech is processed by the brain (Luo et al., J. Neurophysiology, 2006; Luo et al., J. Neurophysiology, 2007).
2) Demonstrating similarities in how speech and complex non-speech sounds are processed by the brain (Xiang et al., IEEE EMBS Conference in Neural Engineering, 2005).
3) Describing a class of abstract auditory computations, the processing of sounds which vary in their interaural correlation (how similar and dissimilar the sounds are at each of our ears) (Chait et al., J. Neuroscience, 2005; Chait et al., Cerebral Cortex, 2006; Chait et al., J. Neurophysiology, 2007). The first reference was the first study to show a direct link between the neural processing of interaural correlations and our perception of those same acoustic features.
4) Demonstrating that the brain uses one neural mechanism to determine when we become aware of new complex sound entities (or objects), and a completely different one when we become aware that a complex sound entity is no longer present (Chait et al., J. Neuroscience, 2007).
5) Showing that direct links can be seen between our behavioral ability to keep track of a simple auditory object (e.g. a person talking) in a complex environment (e.g. at a cocktail party) and the neural processing of the actual tracking (Elhilali et al, submitted). Demonstrating these direct associations at all, between human (or animal) behavior, and the neural processes that govern them, is something of a holy grail in neuroscience.
Much of this research was done in collaboration with David Poeppel. In these collaborations, I push forward the use of MEG in a new direction, as a tool to study the computational principles at work in human auditory cortex, leveraging David’s expertise in cognitive neuroscience.
Unlike the experiments described above, which record MEG signals from the entire brain, investigation at the network level uses the opposite tactic: recording only from specific brain areas. Using electrodes implanted inside the brain of an animal (a ferret), the precise location of the neurons is known, but which neural computations they perform (e.g. which features of a sound are encoded, and how) are not. A great advantage of knowing which neuron it is, and what its neighboring neurons are doing at the same time, is that one can see the neural computations being performed by the neuron as a member of a neural network, where each neuron has a role in the computation. Simon et al. (Neural Computation, 2007) and Depireux et al. (J. Neurophysiology, 2001) showed that the neural computations performed in one area of the brain, primary auditory cortex, are constrained in how they process sound features in a very symmetric way. The most fascinating and informative aspect of this symmetry is that it is a symmetry in time, not in space (or in spectrum). Symmetries in time are very difficult to arrange, since they must be enforced over the entire neural network. In turn, however, they tell us a great deal about how the neural computation is being performed, since the possible network properties are highly constrained by the symmetry.
Much of this research was done in collaboration with Shihab Shamma. In these collaborations, I drove the conceptual development of the ideas above, and developed the analytical methods necessary to tie them to the experimental data.
The brainstem, the part of the brain just above the spinal cord, is where some of the most basic and fundamental auditory computations are performed. One set of computations involves the careful comparison of sounds arriving at the two ears at different times, depending on the physical location of the object making the sound. This interaural time difference computation is especially prominent in the barn owl, but also important in other birds, including chicken and emu.
The third major component of my research program is the theory and modeling of these computations performed in the bird auditory brainstem. I have created biophysically detailed models of how coincidence detection, the co-arrival (or not) of the neural signals encoding the sound from each ear, allows the computation of the location of the sound source (Grau-Serrat et al., Biological Cybernetics, 2003). This model has been used to test, and rule out, possible theories of coincidence detection.
This work is in collaboration with Catherine Carr. In these collaborations, I lead the computational modeling effort and direct how experimental design interacts with the computational model (both informing the model and testing predictions of the model), thus greatly expanding the breadth and depth of the lab’s experimental results.
New techniques or theoretical frameworks are sometimes discovered as part of the research process. For me, these ideas have led to several new methods of analyzing and extracting neural signals (Simon & Wang, J. Neuroscience Methods, 2005; Ahmar et al., IEEE EMBS Conference in Neural Engineering, 2005; Ahmar and Simon, IEEE EMBS Conference in Neural Engineering, 2005; de Cheveigné et al., ICASSP, 2007; de Cheveigné & Simon, J. Neuroscience Methods, 2007), and a new theoretical framework that forges a connection between sound localization and self-initiated movement (Aytekin et al., Neural Computation, 2008).
Before my current research career in computational and experimental neuroscience, I had an earlier career in theoretical physics. My physics training and background has provided me both with a powerful set of mathematical tools, and with the knowledge of when (and when not) to use them. While I no longer pursue this line of research, I am proud of my accomplishments there.