SoundSight uses a cross-species, multi-level approach to study how vision shapes the development of auditory inputs to the occipital cortex.  It will test the hypothesis that during early postnatal development, an extensive network of crossmodal axonal projections is laid out between related brain regions that is mostly pruned away through activity of feedforward inputs of the dominant modality.

Project Description and Objectives

Despite the fact that what we see, hear, feel, taste and smell is processed separately by our senses, we perceive our environment as a unified and coherent whole. This is because brain regions that process the inputs from different senses are interconnected and provide contextual information to each other, allowing multisensory integration, which creates a better representation of the world than what is achieved by each sense separately. How these crossmodal inputs develop and are fine-tuned by experience has not been studied extensively. Experiments in the Collignon lab with congenitally blind human subjects have revealed that some visual cortical regions start responding to auditory inputs. For example, regions that normally respond to visual motion are selectively activated by auditory motion. In parallel, work from the Nieto lab showed an exuberancy of intracortical developmental axonal projections. We hypothesize that during early postnatal development, an extensive network of crossmodal axonal projections is laid out between related brain regions that is mostly pruned away through activity of feedforward inputs of the dominant modality. The remaining crossmodal connections allow for multisensory integration in the adult cortex. In the congenitally blind, the crossmodal connections are not pruned, but strengthened, and become drivers of the network. While this plasticity improves the non-deprived senses, it may interfere with the restoration of vision later in life. In this proposal, we will test this hypothesis and its implications for the restoration of vision in the blind from the level of individual synapses in mice to the level of perception in humans. 

We will study the effects of visual experience during development on functional crossmodal connectivity in visual cortex using newly developed tracing approaches and chronic calcium imaging in mice and functional magnetic resonance imaging (fMRI) studies in human subjects. The functional anatomy of crossmodal connections will be mapped down to the level of dendritic integration in different excitatory and inhibitory neuronal subtypes, using RNAseq and electrophysiological and imaging approaches in mice. We will validate these findings in human subjects with ultra-high-field fMRI and intracranial electrophysiological recordings in humans. We will examine the effects of visual deprivation on the development of the visual system’s functional anatomy and use a neuronal modeling approach to understand how crossmodal innervation and visual deprivation affect neuronal dynamics in visual cortex. Finally, we aim to understand the mechanisms underlying the regulation of crossmodal plasticity in mice, and use this information to explain long lasting effects of early visual deprivation in human subjects and mice and take initial steps to improve restoration of vision at a later age.

SoundSight will be highly complementary to the research performed within HBP. An important mission of HBP is to understand how the brain represents stimuli in their spatiotemporal context and to comprehend how the underlying neural circuits are organized at different levels, in space and time. SoundSight will contribute to these goals by providing novel insights into the development of the neural circuits that allow the brain to integrate sensory inputs, from the molecular level in mice to the functional and network level in humans. The project will benefit from models developed and data acquired within HBP to identify the neuronal cell types that form crossmodal connections, to interpret human fMRI data and to model how experience drives the development of crossmodal connections.


Chronic imaging of calcium responses in visual cortex neurons


Developmental axonal projections from auditory cortex to other brain regions

Partnering Organisations




Christiaan Levelt

Christiaan Levelt is an expert in the field of visual cortex development and plasticity. He heads the Molecular Visual Plasticity Department at the Netherlands Institute for Neuroscience (Amsterdam, the Netherlands), and has studied the mechanisms regulating visual plasticity during development, and more recently in adulthood. He is partner in HBP in which he studies interactions between feedforward and feedback connections in visual perceptual learning, using chronic in vivo two-photon calcium imaging and in vivo and in vitro electrophysiology, combined with opto- and chemogenetics.


Jorrit Montijn

Jorrit Montijn from the Department of Cortical Structure and Function at the Netherlands Institute for Neuroscience has a unique combination of experience with computational modelling of multi-areal circuits, psychophysical and neurophysiological cross-modal experimental work and advanced data analytic approaches. As such, Dr. Montijn is the ideal partner to lead the computational modelling effort and integrate the theoretical modelling results with the experimental work.


Marta Nieto 

Marta Nieto is Chair of the Department of Cellular and Molecular Biology at the Centro Nacional de Biotecnología - Spanish National Research Council (CNB-CSIC) (Madrid, Spain). She is an expert in the development and molecular specification of cortical networks. Her research has focused mainly on the roles of two transcription factors, Cux1 and Cux2, in the developmental plasticity of circuits of the upper cortical layers and the implications of these processes for neurodevelopmental disorders. Recently her research expanded to investigate related mechanisms of sensory-dependent wiring governing also other cortical populations. Her lab expertise includes in utero electroporation, imaging, retrotracing, generation of knock-in mouse alleles using CRISPR/Cas, FACsorting and RNAseq.


Olivier Collignon

Olivier Collignon is the head of the Crossmodal Perception and Plasticity lab at the Université Catholique de Louvain in Belgium. He was awarded an ERC starting grant aiming to understand the functional consequences of sight deprivation and restoration. His research relies on the respective advantages of a plurality of methods to converge toward a comprehensive understanding of the mechanisms underlying crossmodal perception and plasticity. His research is driven by the strong conviction that the study of sensory deprived individuals represents an excellent model to probe how the brain develops, maintains and changes its functional tuning to adapt its interaction with the environment. 



Fazilaty H, Rago L, Kass Youssef K, Ocaña OH, Garcia-Asencio F, Arcas A, Galceran J, Nieto MA. A gene regulatory network to control EMT programs in development and disease. Nat. Commun. 2019; 10, 5115

Rodríguez-Tornos FM*, Briz CG*, Weiss LA, Sebastián-Serrano A, Ares S, Navarrete M, Frangeul L, Galazo M, Jabaudon D, Esteban JA and Nieto M. Cux1 enables inter-hemispheric connections of layer II-III neurons by regulating Kv1-dependent firing. Neuron 2016; 89, 494.

Montijn JS, Meijer GT, Lansink CS, Pennartz CM. Population-Level Neural Codes Are Robust to Single-Neuron Variability from a Multidimensional Coding Perspective. Cell Rep. 2016; 16, 2486.

Van Ackeren M, Barbero F, Mattioni S, Bottini R, Collignon O. Neuronal populations in the occipital cortex of the blind synchronize to the temporal dynamics of speech. Elife 2018; e31640

Benetti S., van Ackeren M.J., Rabini G., Zonca J., Foa V., Baruffaldi F., Rezk M., Pavani F., Rossion B. and Collignon O. Functional selectivity for face processing in the right temporal “voice” area of early deaf individuals. PNAS 2017; 114, E6437

De Heering A., Dormal G., Pelland M., Lewis T., Maurer D., & Collignon O. A brief period of postnatal visual deprivation alters the balance between auditory and visual attention. Curr. Biol., 2016; 26, 3101

Sommeijer JP*, Ahmadlou M*, Saiepour MH, Seignette J, Min, Rm Heimel JA, Levelt CN. Thalamic inhibition regulates critical period plasticity in visual cortex and thalamus. Nat. Neurosci. 2017; 20, 1715

Smit-Rigter L*, Rajendran R*, Silva CAP, Spierenburg L, Groeneweg F, Ruimschotel EM, van Versendaal D, van der Togt C, Eysel UT, Heimel JA, Lohmann C, Levelt CN. Mitochondrial dynamics in adult visual cortex are limited and not affected by axonal structural plasticity. Curr Biol, 2016; 26, 2609.



Key facts

Time frame: 2020 - 2022

Origin: FLAG-ERA JTC2019