As a highly automated device, robots have entered all aspects of human life and work, such as industrial production, service industry, medical care and other fields.
With the emergence and development of emerging technologies such as AI, human demand for robots has risen from "automation" to "intelligence", hoping that robots can have the general perception ability of humans, and can make intelligent decisions based on perceived information, and get along with humans and cooperate with each other.
The electronic skin can enable the robot to obtain the ability of large-scale multi-perception, provide security for human-machine coexistence, and provide interactive means for human-machine collaboration.
However, the current robot electronic skin has many problems such as single perception function, poor performance, inconvenient use, difficult to expand in a large area, weak interaction ability, etc., which is difficult to meet the actual application needs of intelligent robots.
To this end, Professor Zhu Rong and his team in the Department of Precision Instruments of Tsinghua University have designed and developed a new multifunctional electronic skin with large measurement range, high sensitivity and fast response by using their own original distributed piezoelectric sensing principles. The electronic skin realizes the multi-modal perception of "contact force, contact position, proximity and temperature", combined with the intelligent interaction control strategy they propose, realizes robot safety prevention and control, human-computer interaction and collaboration, and can be widely used in the field of intelligent robots.
The research paper was published in Advance Science under the title "Multifunctional Electronic Skins Enable Robots to Safely and Dexterously Interact with Human".[1] With Rong Zhu as corresponding author, The team's doctoral students Li Guosheng and postdoctoral fellow Liu Shiqiang are co-first authors.
Figure | Related research papers (Source: Advanced Science)
Previously, in order to solve the problem of flexible multi-sensor integration, the team proposed "new theory of pressure sensing and new mechanism of multimodal sensing based on thermal induction", realized the integrated measurement of pressure, temperature, thermal physical properties and other parameters, and developed a multifunctional sensing flexible electronic skin.
The research paper was published in Advance Materials [2] in 2017 under the title Electronic skin with multifunction sensors based on thermosensation[ 2].
Figure | Photo of Zhu Rong's research team (Source: Zhu Rong)
Subsequently, the team developed a flexible multi-dimensional haptic sensor based on the multilayer stacking design of thermodynamic functional material and thermal film in view of the application needs of intelligent robots for fine tactile perception of terminal manipulators.
The paper was published in Science Robotics in 2020 under the title Skin-inspired quadruple tactile sensors integrated on a robot hand enable object recognition[3].
This flexible multi-dimensional haptic sensor realizes the in-situ integration of multiple perceptions (touch, temperature, object, etc.), and can be applied to the robot's dexterous hand perception system, which can identify the material, shape and temperature of items with high precision, and successfully complete the classification task of seven kinds of garbage.
This time, the research team further oriented to the application needs of human-computer interaction for intelligent perception and intelligent measurement and control, and developed a new type of robot electronic skin with large-format multi-touch perception, which solves the problems of safety prevention and control and soft interaction control in human-computer collaboration applications, and is expected to greatly improve the efficiency and intelligence of robot task execution.
Figure | Prototype and Sensing Principles for Novel Multifunctional Electronic Skins (Source: Advanced Science)
Different from the conventional electronic skin realized by using high-density array sensing units, this new multifunctional electronic skin uses only 4 distributed piezoelectric sensing units to achieve force sensing and contact position sensing of large-covered electronic skins, and it also integrates temperature sensing and proximity sensing functions, which can realize safety early warning and prevention before human-machine collision and ensure the safety of human-computer interaction.
The researchers pointed out that due to the large range (500N), high sensitivity (0.5N), fast response (0.09s) and strong reliability of the electronic skin's haptic perception, the use of electronic skin to control the robot is very flexible and convenient. At the same time, it also shows excellent scalability, and can adapt to different shapes of robotic arms, which can meet the needs of different types of robots.
(Source: Advanced Science)
The researchers also proposed a new type of human-computer interaction intelligent control strategy, combined with the multimodal perception performance of this new electronic skin, the robot can successfully complete automatic obstacle avoidance, collision detection and safety control, flexible human-computer interaction and teaching reproduction and other tasks, and show the actual effect of teaching robots to play Morse code, handwriting, and tai chi.
It is understood that after wearing this electronic skin on the surface of the robot body, it is easy to control the robot by touching the electronic skin by human hands, and realize the flexible teaching of the human machine and the transmission of action intentions. Compared with the traditional programming teach-in and current feedback drag-and-teach, this electronic skin can improve the safety, flexibility, accuracy and execution efficiency of robot teach-in.
Figure | Applying new multifunctional electronic skins to robots' human-computer interaction tasks, such as teaching robots to play Morse codes (Source: Advanced Science)
The team said that this is a multidisciplinary and interdisciplinary systematic work, involving advanced materials, micro-nano electromechanical, intelligent control and artificial intelligence and other fields of expertise, the knowledge and professional skills of researchers have high requirements. The Department of Precision Instruments of Tsinghua University is the cradle for cultivating system-level professionals, and the subject areas cover many aspects such as optoelectronic measurement and control.
In addition, the smooth development of this research work is inseparable from the support of the key projects of the National Natural Science Foundation of China, the National Key Research and Development Program and the key projects of the Beijing Municipal Natural Science Foundation.
Figure | Zhu Rong (Source: Zhu Rong)
Zhu Rong said, "This research provides an effective solution for promoting human-computer safe interaction and intelligent collaboration, which is expected to accelerate the intelligent application process of robots in intelligent manufacturing, intelligent services, medical and health and other fields." ”
In the future, the team will be committed to developing a robot system with full tactile perception ability from the end of the touch to the electronic skin on the surface of the body, and combine the tactile perception with visual perception and auditory perception to establish an intelligent robot system with the multidimensional perception fusion of "sight, touch and hearing" and study its intelligent application in complex tasks or scenarios.
-End-
reference:
1.Guozhen Li, Shiqiang Liu, Qian Mao, Rong Zhu*, Multifunctional electronic skins enable robots to safely and dexterously interact with human. Advanced Science, 2022, 2104969.
2.Shuai Zhao, Rong Zhu*, Electronic skin with multifunction sensors based on thermosensation,Advanced Materials. 2017, 29(15), 1606151.
3.Guozhen Li, Shiqiang Liu, Liangqi Wang, Rong Zhu*, Skin-inspired quadruple tactile sensors integrated on a robot hand enable object recognition,Science Robotics. 2020, 5, eabc8134.