Researchers of the Human Brain Project at University of Amsterdam and University of the West of England have built a neural network architecture on the EBRAINS research infrastructure that can enable robots to effectively combine multiple senses for perception and navigation. The team, led by Prof. Dr Cyriel Pennartz, has now published the findings in the open-access journal Frontiers in Robotics and AI.
One of the basic challenges for robots to autonomously navigate their environment is the ability to recognize familiar places. Places can be recognized most accurately if information from different senses is combined. While it's relatively easy for humans and animals to do this and use the information to move through the world efficiently, artificial steering systems so far still struggle with this task.
The researchers now show that an artificial neural network based on real-life brain data from rats (“MultiPredNet”) can combine visual and tactile information more robustly than previous artificial systems. The algorithm is able to recognize places even when one of the sensors - visual or tactile – momentarily ‘drops-out’, akin to the way humans or animals are able to navigate their environment even with incomplete or changing information from the different senses.
Connected to the biomimetic “Whiskeye”-robot, the network enables effective place recognition of its environment and lets the robot efficiently navigate through its surroundings.
Whiskeye, a robot with cameras for eyes and 24 artificial whiskers that collect tactile information
The algorithm is extensible, so that further senses can be included in the future. Both the robot and environment have been replicated virtually on the EBRAINS Neurorobotics Platform, where they are available for further open testing.
The system provides a path to use neurobiologically inspired deep learning for improving navigation systems in technology.
Read the full paper in Frontiers in Robotics and AI: Multimodal Representation Learning for Place Recognition Using Deep Hebbian Predictive Coding
Discover the EBRAINS Neurorobotics Platform.