Our goal in SP3 is to uncover the neural mechanisms underlying cognitive processes, such as learning, perception, sleep, consciousness, and associated systems phenomena.

The results from our research provide the framework for the development of computational models of cognitive and systems-level processes, which can be implemented in robots and neuromorphic computing systems. We address these issues at multiple levels (cells, groups, networks, brain systems) and work to unify different disciplines.

How does the brain create a representation of an object from multisensory information? Imagine an apple — its greenness, sour taste and fresh smell; how does the brain create an invariant representation from these multiple sensory inputs? This question is crucial since these representations are the basis for higher cognitive processes such as category formation, reasoning and language. One of our goals is to develop a "deep learning" neuronal network that learns to recognize objects and functions in a way similar to real neurobiological systems.

One of the deepest unsolved problems in science is the nature of consciousness - how is consciousness generated by the brain? There are several clinical and ethical problems that limit how we can address this question, such as assessing the level of consciousness in patients following brain injury. Novel ways to measure consciousness levels will make clinicians less dependent on purely behavioral measurements; this will benefit, for example, coma patients. Similarly, how can disparate phenomena such as sleep and wakefulness emerge from the same cortico-thalamic systems in the brain? To answer this question, we will investigate slow-wave activity and simulations of large populations of firing neurons in mice and humans.

Finally, we investigate brain mechanisms of memory. Episodic memory, pertaining to our personal, conscious experiences set within space and time, defines who we are. The brain's ability to recall objects and experiences from multisensory information (e.g. vision, hearing or touch) is key to understanding human memory. We are conducting a coordinated series of experiments to identify the neuronal mechanisms behind episodic memory, and validate them by computational models and robotic systems. Again, hierarchical cortex-like models play an important role, but this time these are focused on the processing of visual and tactile properties of objects.

Our cross-disciplinary approach, which cuts across multiple levels of neural and brain organization, will work to elucidate mind-brain relationships that have previously been all too elusive. In doing so, we will closely collaborate with other Subprojects.

The projects and partners stated below were selected as the best out of 57 research consortia across Europe who competed together to form a new Subproject on Systems and Cognitive Neuroscience. This new Subproject succeeded the first subproject 3 on Cognitive Architectures. The work of the new Subproject 3 started in April 2016, under the Specific Grant Agreement Phase 1 (SGA1).


SP Leader: Cyriel PENNARTZ

Deputy SP Leader: Johan Frederik STORM

Work Package Leaders:

  • WP3.1 Context-sensitive Multisensory Object Recognition: Lars MUCKLI
  • WP3.2 Wave Scaling Experiments and Simulations: Pier Stanislao PAOLUCCI
  • WP3.3 Episodic Memory as Multisensory Reconstruction: Cyriel PENNARTZ
  • WP3.4 Experimental and Computational Exploration of Consciousness Mechanisms and Methods in Mice and Humans: Johan Frederik STORM
  • WP3.5 Scientific Coordination, Project Management and Communication: Cyriel PENNARTZ
  • WP3.6 SP3 Contributions to Co-Design Projects and Infrastructure: Cyriel PENNARTZ


Publication highlights (since 2016):

Gosseries, O., Pistoia, F., Charland-Verville, V., Carolei, A., Sacco, S., & Laureys, S. (2016). The Role of Neuroimaging Techniques in Establishing Diagnosis, Prognosis and Therapy in Disorders of Consciousness. The Open Neuroimaging Journal, 10:52-68.

Izquierdo-Serra, M., Bautista-Barrufet, A., Trapero, A., Garrido-Charles, A., Díaz-Tahoces, A., Camarero, N., Pittolo, S., Valbuena, S., Pérez-Jiménez, A., Gay, M., García-Moll, A., Rodríguez-Escrich, C., Lerma, J., de la Villa, P., Fernández, E., Pericàs, M.À.,Llebaria, A. & Gorostiza, P. (2016). Optical control of endogenous receptors and cellular excitability using targeted covalent photoswitches. Nature Communications, 7: 12221.

Lansink, C.S., Meijer, G.T., Lankelma, J.V., Vinck, M.V., Jackson, J.C., & Pennartz, C.M.A. (2016). Reward expectancy strengthens CA1 theta and beta band synchronization and hippocampal-ventral striatal coupling. Journal of Neuroscience, 36: 10598-10610.

Montijn, J.S., Meijer, G.T., Lansink, C.S., & Pennartz, C.M. (2016). Population-level neural codes are robust to single-neuron variability from a multidimensional coding perspective. Cell Reports, 16: 2486-2498.

Montijn, J.S., Olcese, U., & Pennartz, C.M.A. (2016). Visual stimulus detection correlates with the consistency of temporal sequences within stereotyped events of V1 neuronal population activity. Journal of Neuroscience, 36: 8624-8640.

Murphy, S.C., Palmer, L.M., Nyffeler, T., Müri, R.M., & Larkum, M.E. (2016). Transcranial magnetic stimulation (TMS) inhibits cortical dendrites. eLife, 5: e13598.

Nieminen, J.O., Gosseries, O., Massimini, M., Saad, E., Sheldon, A.D., Boly, M., Siciari, F., Postle, B.R., & Tononi, G. (2016). Consciousness and cortical responsiveness: a within-state study during non-rapid eye movement sleep. Scientific Reports, 6: 30932.

Olcese, U., Bos, J.J., Vinck, M., Lankelma, J.V., Van Mourik-Donga, L.B., Schlumm, F., & Pennartz, C.M.A. (2016). Spike-based functional connectivity in cerebral cortex and hippocampus: loss of global connectivity is coupled to preservation of local connectivity during non-REM sleep. Journal of Neuroscience 36: 7676-7692.

Versendaal, D., & Levelt, C.N. (2016). Inhibitory interneurons in visual cortical plasticity. Cellular and Molecular Life Sciences, 73: 3677-3691.


SP3: Cognitive Architectures - Ramp-Up Phase (October 2013 - March 2016)

In the Ramp-Up Phase, the Subproject on Cognitive Architectures analysed the architecture for several cognitive functions, e.g.: perception-action; motivation, decision and reward; learning & memory; space, time & numbers; sensory and multimodal perception; and, other characteristic capabilities of the human brain. We developed localiser protocols for these functions, and acquired unique benchmark data for modellers to work with SP4 (Theoretical Neuroscience) to translate these into cognitive models. These focused on two theoretical models, cognitive architectures for spatial navigation and visual action recognition.

Publication highlights (Ramp-Up Phase):

Ben-Yakov, A., Rubinson, M., & Dudai, Y. (2014). Shifting gears in hippocampus: temporal dissociation between familiarity and novel signatures in a single event. Journal of Neuroscience, 34: 12973-12981.

Blanke, O., Slater, M., Serino, A. (2015). Behavioral, neural, and computational principles of bodily self-consciousness. Neuron, 88: 145-166.

Dehaene, S., Meyniel, F., Wacongne, C., Wang, L., & Pallier, C. (2015). The neural representation of sequences: from transition probabilities to algebraic patterns and linguistic trees. Neuron, 88: 2-19.

Frégnac, Y., & Bathellier, B. (2015). Cortical correlates of low-level perception: from neural circuits to percepts. Neuron, 88: 110-126.

Fries, P. (2015). Rhythms for cognition: communication through coherence. Neuron, 88: 220-235.

Gabitov, E., Manor, D., & Karni, A. (2015). Patterns of Modulation in the Activity and Connectivity of Motor Cortex during the Repeated Generation of Movement Sequences. Journal of Cognitive Neuroscience, 27(4): 736-751.

Meyniel, F., Schlunegger, D., & Dehaene, S. (2015). The Sense of Confidence during Probabilistic Learning: A Normative AccountPLoS Computational Biology11(6): e1004305.