Co-Design Project 4

Co-Design Project 4 -
Visuo-Motor Integration

The overall aim of Co-Design Project 4 is to develop and implement multi-modal, neurobiologically realistic models of sensorimotor integration, to include advanced object recognition and spatial localisation tasks to guide robotic motor control.

To do this, the CDP will use a top-down modelling approach to translate conceptual models into computational architectures and ultimately into spiking neuronal network models that both use the HBP Research Infrastructure and contribute to its co-development.

CDP4 will initially focus on visuo-motor integration, which will include modelling of the attention-for-action tasks that are required to obtain parameter specifications for motor execution. This will integrate ongoing work from SP2 (Strategic Human Brain data) and work from SP3 (Cognitive Architectures) on realistic models of invariant object recognition and the role of attentional selection for both object recognition (ventral processing stream) and motor planning/spatial awareness (dorsal processing stream). These models are empirically constrained and will be combined with bottom-up simulations developed in collaboration with SP4 (Theoretical Neuroscience), SP6 (Brain Simulation) and SP7 (High Performance Analytics and Computing); these will lead to empirically validated computational architectures of visuo-motor integration. In later stages of the HBP, the models will be extended to other sensory modalities, focusing on the somatosensory and auditory systems. The sensorimotor modelling will be integrated with the development of algorithms for multi-modal guidance of robotic motor control with feedback (somatosensory) and feed-forward (visual and auditory) loops, in collaboration with SP10 (Neurorobotics).

The combined efforts of the CDP4 group could lead to an HBP showcase of neuroscience observations implemented in the HBP Research Infrastructure. This could demonstrate to the scientific community the exciting work that cuts across the entire Project and helps to integrate the work of different SPs. For example, CDP4 may show how visual and somatosensory information is integrated in the human brain to continuously adjust motor control during complex tasks such as writing or using a scalpel. By establishing a human brain atlas at ultra-high resolution, and unravelling the corresponding computational circuits by large-scale simulations, we can incrementally develop a truly comprehensive visuo-motor and somatosensory brain model of complex motor control. Subsequently translating such models to neurorobotic applications can provide an end product that can have real-world applications, such as robotic surgery systems.

The CDP4 Products to be developed are as follows:

  • CDP4-P1: Neural sensorimotor integration network
  • CDP4-P2: Large-scale models on sensorimotor integration
  • CDP4-P3: Spiking neuron models for sensorimotor integration
  • CDP4-P4: Neurorobotic closed-loop engine

CDP Science Leader: Rainer GOEBEL

CDP Implementation Leader: Sonja GRÜN