How does the face recognition system in the mobile phone APP authenticate that a person in front of the camera is a living person?
For the average person, it's simple. But for special groups such as people with facial paralysis, facial burns, and the elderly with disabilities and dementia, this is a problem, especially because they are unable to blink autonomously or coordinatedly, and become "digital refugees".
This is not a niche group, just facial paralysis, the annual incidence rate in the mainland exceeds 10 million.
Face recognition technology is developing rapidly and blooming everywhere, and it is widely used in remote identity verification on mobile phones, which is convenient for people's lives. How to make these special groups keep up with the pace of the times, the Red Star News reporter found that the exploration of breaking through the predicament has begun.
A disabled (demented) elderly person
Paralysis
"More than 10 million cases occur in the country every year"
In early winter, the temperature in the Sichuan Basin plummeted. Under the cold wind, Ms. Li was paralyzed.
On the morning of December 6, when Ms. Li was brushing her teeth, she noticed that the left corner of her mouth was inadvertently leaking. At the same time, she felt some numbness in her face, so she studied it in the mirror: "I tried to smile, tried to puff up with my mouth, but only the right side of my face could move, and my mouth was crooked. I think I may have facial paralysis. “
After another day, Ms. Li found that she could not control her left side of her brow and that her left eye was difficult to close. Subsequently, Ms. Li went to a hospital to see a doctor and was diagnosed with peripheral facial paralysis: "The doctor said that it was caused by infection, which may have something to do with the cold wind blowing through the window while driving, and I often stayed up late during that time, so my immunity may not be very good." ”
The facial paralysis on the left side of her face made it difficult for Ms. Li to use the facial recognition function in her mobile phone. "A few days ago, I used an APP to check social security, and when logging in, I need to brush my face, blink and shake my head to pass the authentication, my left eye has been uncontrolled, and the system prompts that the operation is timed out. Ms. Li said that after repeated attempts, she pulled the eyelid of her left eye with her hand, and the system told her to "cover her face", and after more than ten minutes of continuous attempts, she still failed to pass the certification, so she could only go offline to handle social security business.
Blinking uncoordinatedly, resulting in face recognition authentication failure
There are also often netizens who complain about this problem. There are often netizens on the online platform who say: "SOS! When did face recognition consider the feelings of patients with facial paralysis?" Hefei Traffic Radio also reported that a takeaway brother was suspended because he could not recognize faces because of facial paralysis, and the takeaway platform arranged a special person to assist him after learning about it.
Screenshot of related news reports
He Liu, an acupuncture pain doctor at Chengdu First People's Hospital, told reporters that in the theory of Western medicine, peripheral facial paralysis is usually infected by a virus, which invades the facial nerve, thus causing a series of neurological symptoms. Traditional Chinese medicine believes that peripheral facial paralysis is a physical lesion caused by factors such as staying up late, blowing cold wind, and weak righteous qi, which is often referred to as "stroke".
He Liu told reporters that it is common to be unable to blink normally after peripheral facial paralysis: "The virus invades the facial nerve, and the brain cannot send instructions to the muscles around the eyes unobstructed, and its symptoms range from mild to severe." ”
He Liu also introduced that the incidence of facial paralysis is not a small number, there are more than 10 million people with facial paralysis every year in the country, and bus drivers and taxi drivers who often blow the wind are a high-incidence group of facial paralysis, and their recovery period is several weeks to more than half a year.
burn
"The digital system doesn't know me anymore"
"When I was very young, the neighbor's child was naughty and pushed me into the fire pit, and the trajectory of my life changed forever. ”
That tragedy left Song Wenjiao severely burned his face and lost ten fingers. But the cruel reality did not knock him down, and the 43-year-old is the founder of the Xuzhou Hantang Public Welfare Development Center, which provides free vocational skills training such as e-commerce, handicraft production, and pastry for the disabled, and provides rehabilitation exercises for the disabled. By building an in-store service platform for helping the disabled, more than 1,000 disabled people have been driven to work and start businesses. He has also won many awards for this, and was also rated as a five-star charity volunteer by the Xuzhou Charity Federation.
Song Wenjiao's team provides skills training for people with disabilities
The mobile Internet has changed everyone's lives, the deaf and mute can also communicate with sign language through video chat, and the blind can also use visual assistance to navigate and guide the way, but there are still some people with disabilities who are stuck in the dilemma of face recognition. Song Wenjiao introduced that in addition to patients with facial burns like himself, patients with Down syndrome, facial spasms, cerebral palsy and people with other facial defects often encounter the problem of not being able to recognize and authenticate faces. When using bank, human resources and social security, and medical insurance apps, it is difficult for them to blink and open their mouths independently or coordinatedly, and their teams need to take them to offline counters to handle it.
"I once missed the high-speed rail because I couldn't recognize my face, and I had no choice but to go to the counter. Song Wenjiao said that for people with disabilities who cannot perform face recognition, many airports, high-speed railways, and stations provide access for special people, but not every station has such a passage, so their travel will have certain limitations.
Because of the burns, Song Wenjiao's eyes are asymmetrical, so he will encounter difficulties when carrying out face recognition authentication. "I have the impression that after 2018 and 2019, banking apps began to widely use face recognition authentication, which became more and more difficult for us, and we could only go offline to handle business such as transfers. He was deeply helpless about this. He hopes that apps such as banks, human resources and social security, and medical insurance can open special channels for special groups of people and carry out special identification.
Similar to Song Wenjiao's experience, there is also Qiao Yan, a girl from Mount Emei, who was disfigured in 2019 due to a spontaneous combustion accident in a van. She told reporters that because she can't use face recognition, she is now completely unable to use banking apps. When passing through some access control and security gates, it is often impossible to pass face recognition authentication, and sometimes it is necessary to take off the hat and verify it repeatedly. When she took off her hat and showed a large area of scarring, she felt tremendous psychological pressure: "These digital systems don't know me anymore. It is hoped that the society can strengthen the research and optimization of face recognition technology, so that each of us can enjoy the convenience brought by technology. ”
Professor Tang Yong, president of Chengdu Badachu Medical Aesthetic Hospital, told reporters that when our brain sends an electrical signal that we need to blink, the orbicularis oculi muscle contracts and drives the skin, but the skin of patients with facial burns is a scar, and its elasticity and mobility are very poor, and some will be accompanied by contraction spasms, making the blinking speed slow or the eyelids are not fully closed.
Tang Yong introduced that whether patients with facial burns can blink in coordination is also related to the location, scope and degree of burns. If the burn does not injure the muscles, it can usually be repaired medically to re-establish the ability to blink. And if you hurt your muscles, you may be affected in the blink of an eye for the rest of your life.
Disability, dementia
"Grandpa is trapped in the time limit of facial recognition"
"Look at the ceiling, look at the ground, look at the ceiling, look at the ...... on the ground"
On December 10, Xiao Cong, a citizen of Chengdu, repeatedly said such a thing with his family, and they were facing his 83-year-old grandfather Dong Hai. On that day, the family needed Dong Hai to apply for long-term care insurance through the medical insurance APP, and an accidental fall a few years ago caused Dong Hai to suffer from a cerebral hemorrhage, and since then he has become a disabled elderly person and needs to be accompanied by a nurse. Therefore, no matter how patiently the family guided, Dong Hai still couldn't blink smoothly and couldn't pass the face recognition authentication of the APP.
Satoshi and his family help his grandfather with face recognition
"We held up the phone and asked Grandpa to blink, but Grandpa was always very slow to react, either open for a long time or closed for a long time. Xiao Cong said: "And there is a time limit for face recognition authentication, about ten or twenty seconds, and grandpa has never been able to blink in a short period of time and is trapped in the time limit." That day, Xiao Cong and his family tried repeatedly like this for almost an hour, but Dong Hai still failed to authenticate, and "his mentality exploded".
In the ward of the Department of Neurology (Alzheimer's Disease Comprehensive Management Center) of Chengdu Eighth People's Hospital, there are many elderly people like Dong Hai. On December 20, in an activity room in the ward, several nurses took care of 13 elderly people, and two doctors walked back and forth to carefully examine the elderly. Towards the end of the examination, the doctor showed a video of finger exercises on TV, but only 3 or 4 of the 13 elderly people could follow the video to do finger exercises. They are either disabled, demented, and generally need long-term care insurance, which must be done with the assistance of a nurse.
Doctors examine the elderly
How to apply for long-term care insurance for the disabled and dementia elderly at home? Two insurance company staff told reporters that if the elderly cannot pass the face recognition certification, they need to apply offline.
Ms. Zhou, the head nurse of the ward, told reporters that the disabled and dementia elderly people are unable to take care of themselves, unable to manage their own behavior, and difficult to communicate normally. Older people with Alzheimer's disease have poor memory and often hallucinate. Therefore, the most important thing in care is to protect their safety and carry out comprehensive daily care, which requires a lot of patience on the part of medical staff.
Ms. Zhou told reporters that when they first applied for long-term care insurance, the insurance company would send someone to the hospital to handle it, avoiding the process of blinking during face recognition. Next, the elderly also need to carry out "survival certification" every month, and also need face recognition, at this time, the nurse will hold the mobile phone, record 7 or 8 seconds of video upload for the elderly, and sometimes the elderly do not cooperate, they need to be patient and repeatedly recorded.
Focus 01: Why blink an eye for face recognition?
Lei Zhen, a researcher at the Institute of Automation of the Chinese Academy of Sciences, said: "It is for live verification, and it is confirmed by blinking that a living person is standing in front of the camera, not a photo. There are also silent liveness detection methods, which directly judge whether it is a real person based on the content and quality of the image, and do not require the user to blink and other cooperative actions, and silent liveness detection is generally placed on the server side (rather than front-end devices such as mobile phones). ”
Hao Zongbo, associate professor of Chengdu University of Electronic Science and Technology, introduced: "Liveness detection can be used in sensors, for example, some access control systems are used, and they can be detected without the user's cooperation to do relevant actions. Sensors are usually not used on mobile phones, so users are required to blink, open their mouths, and shake their heads, which is the simplest method of liveness detection. ”
According to the technicians of a face recognition R&D company, blinking is a natural and difficult to fake action, and the main consideration for liveness detection through blinking is to improve safety and accuracy.
人脸识别 图据 IC photo
02 Is there a technical path for the mobile phone without blinking?
Lei Zhen said that as far as he knows, R&D institutions have not paid much attention to the plight of the above-mentioned "digital refugees". Hao Zongbo said that R&D institutions will consider the issues of efficiency and market, because minority groups need to invest more energy to develop such technologies.
Hao Zongbo introduced that at present, some people are studying the pure use of cameras to do live detection, but there are not many landings, and its recognition accuracy is not ideal. "For example, if you look at a person with two cameras, you can see that it's three-dimensional, and if you're looking at a photo, it's two-dimensional." However, there are still loopholes in this scheme, and if you wear a thin human skin mask, you can cheat it.
Hao Zongbo said that based on the current mobile phone equipment, he believes that there are technical means to solve the above dilemma. For example, we can consider multi-angle acquisition, "For example, our camera obtains feature information from different angles in front of the left, front, and right to complement each other. ”
In addition, iris recognition also seems to be a solution. In this regard, Hao Zongbo said that iris recognition is a contact recognition method, which requires the eyes to be very close to the recognition device, and its recognition accuracy is very high, usually used in places with high security requirements. However, if it is applied universally, it will incur a higher cost.
Lei Zhen introduced that there are already some solutions, such as some APPs using the method of dazzling live detection, keep it still, and the screen will emit different colors of light to verify whether (the object) is alive.
According to Guangming.com, Gao Li, a senior technical expert at China Mobile Financial Technology Co., Ltd., a subsidiary of China Mobile, introduced a self-developed "colorful liveness detection" technology. The reporter learned from the relevant staff of China Mobile that the business is being improved.
Ding Daoshi, a veteran Internet observer, said that artificial intelligence needs to have an ethical outlook, needs warmth, and needs to illuminate those angles that we have not paid attention to. Face recognition is one aspect of AI applications, which can use more dimensional verification methods to help the above-mentioned special groups verify, such as fingerprints, palm veins, language verification, etc., "There is always one that can." "At present, government affairs, finance and other APPs use a large number of face recognition authentication, it is recommended to take government departments as the guide, technology companies to make a difference, two-pronged, so that more dimensional verification methods can be implemented, illuminating special groups.
policy
A number of initiatives have been taken to improve the way of identification
According to the Chinese government website on December 15, the People's Bank of China, the National Medical Insurance Bureau, and the China Disabled Persons' Federation carefully studied and handled netizens' comments about "facial paralysis patients, disabled elderly people, etc. are hindered from doing things because they cannot 'brush their faces'".
The reply mentioned that the People's Bank of China issued the "Mobile Financial Client Application Software Barrier-free Service Construction Plan" to guide financial institutions to focus on the daily high-frequency financial scenarios of special groups, and create more than 1,100 barrier-free financial apps such as caring version, voice version, and large-print version, so as to continuously improve the availability and satisfaction of financial services. The National Health Insurance Administration replied that at present, 32 provincial medical insurance information platforms across the country have supported the "offline activation of medical insurance code" function, and 426,000 insured people have activated the medical insurance code offline; The China Disabled Persons' Federation replied that it will further intensify its work in the future, cooperate with relevant functional departments to take more effective measures to encourage relevant enterprises to provide more identity verification methods such as audio verification codes and gesture verification codes, so as to facilitate disabled people to handle financial and other businesses and participate in social life.
Note: Dong Hai, Ms. Li, and Xiao Cong are pseudonyms in the article
Red Star News reporter Wang Tuo
Edit into a preface
(Download Red Star News, there are prizes for reporting!)