Inspired by the star-nosed mole rat, the intelligent manipulator can accurately identify buried life in earthquakes!

Tao Hu, a researcher at the Shanghai Institute of Microsystem and Information Technology of the Chinese Academy of Sciences, developed the "Touch-Smell Integrated Bionic Intelligent Manipulator", under the burial scene of the human body covered by rubble piles, the bionic manipulator can identify 11 typical objects, including the human body, and assist in emergency rescue, and the accuracy of touch-smell synesthesia recognition is 96.9%. The study was recently published in Nature Communications.

Visual perception impairment caused by burial accidents such as earthquakes, landslides, and landslides has greatly hindered the search and location of trapped personnel by rescuers. In view of the lack of visual input in extreme environments, in order to ensure the best rescue time and improve the search and rescue and survival rate, Tao Hu's team relied on the national science and technology innovation 2030 "new generation artificial intelligence" major project to conduct research on the identification and rescue needs of trapped people.

The star-nosed mole rat has long lived underground, and its vision and hearing have gradually deteriorated, and it has evolved star-shaped tentacles around the nostrils, subtly concentrating the sense of touch and smell to help it hunt or judge danger. Inspired by the "touch-smell fusion" perception of the star-nosed mole, the team integrated the MEMS olfaction and haptic flexible sensor array with multimodal machine learning algorithms to build an intelligent manipulator that imitates the touch-smell of the star-nosed mole.

Thanks to the silicon-based MEMS gas sensor whose sensitivity exceeds human 1 order of magnitude, and the pressure sensor whose detection limit exceeds that of human beings by 1 order of magnitude, after the bionic manipulator "touches" the object, it can accurately obtain its key features such as local micromorphography, material hardness and overall contour, and the palm can simultaneously sniff out the "fingerprint" odor of the object, and further process it in real time through the bionic touch-smell synesthesia machine learning neural network, and finally complete the identification of the human body, confirm the part, judge the burial state, remove the obstacle, and close-loop rescue.

In cooperation with the Shanghai Fire Research Institute of the Ministry of Emergency Management, the researchers restored the real rescue environment through field research in front-line fire rescue units, so that the bionic manipulator could identify 11 typical objects, including the human body. Compared with the previously published scientific research results, the bionic manipulator only uses one-seventh of the number of sensors to achieve a more ideal identification purpose, and the accuracy of touch-smell synesthesia recognition is 96.9%, which is 15% higher than that of a single feeling. And the reduced sensor size and sample size are more suitable for rapid response and application in complex environments and limited resource conditions. In addition, in the face of the common situation in the actual rescue of the presence of interference gas or partial damage to the device, through the complementarity of multimodal perception and the rapid adjustment of the neural network, the system still maintains a good accuracy rate of more than 80%.