When you drive for 3 hours in a row, the beautiful woman in the co-driver yells at you: "Pay attention, don't fall asleep!" You're tired of living, I don't want to die!"
Your head is instantly agitated and fully awake! You think to yourself: She has been playing with her phone and has not looked at me, why can she notice that I am driving away?
I thought she was just a kind and simple beauty with a small IQ, but why is her driver monitoring algorithm (DMS) so powerful?
As a DMS algorithm engineer, you fall into deep contemplation... Why? Why is your DMS system, which has been cold for ten years and stayed up late thinking, even inferior to a girl who doesn't look at you in the right eye?
The basic structure of DMS is simple: perceived decision execution. It even sounds similar to an autonomous driving system. One of the most complex is perception, let's focus on perception!
First, the perception sensor: the driver's heartbeat, eyes, expressions, posture...
To obtain a person's mental state, professional perception instruments are counted as sleep monitors in hospitals:
According to the comprehensive data of brain waves, emography, skin electricity, ECG, breathing, etc., you can accurately perceive your state: whether you are sleeping deeply, sleeping lightly, or driving seriously.
If you think about it with your heels, you know that although this system is professional, it is impossible to use it on a mass production car. Secondly, some people have proposed to use smart wearable devices to communicate with the on-board system to obtain people's ECG, blood pressure, and body temperature data:
This idea is really good, but the communication between smart wearable devices and car machine systems is not smooth and difficult to implement. If you want to achieve this, wait until Huawei and Apple build mass production cars!
Now, take a step back! Install a camera on the steering wheel to monitor face, limb, and eye movements, and install a torque sensor to monitor hand movements – this is also the current mainstream solution.
As you can see in the following figure, there is a DMS camera on the steering wheel of xiaopeng P7, and a string of small purple lights for infrared fill light and illuminate faces (these small lights are invisible to the human eye, and mobile phones can be photographed).
There are two types of steering wheel sensors that detect the hand:
Torque sensor: If the rotation of the hand is not detected for a long time, it will alarm - when the advanced assisted driving is enabled, it will be more annoying; the hand must not only be on the steering wheel, but also apply a small torque all the time, too easy to exit the auxiliary driving, too small to detect, quite a test of skill. Capacitive HOD sensor: It is easier to detect whether the hand in driving is on the steering wheel without applying torque.
At this point, you begin to think about a problem: when the beauty is yelling at you, she is neither looking at you (the camera is not working) nor sneaking your hand (the steering wheel sensor is not working), so how does she know that you are distracted?
In fact, we can also take a step back: neither cameras nor steering wheel sensors, relying only on the state of the vehicle itself.
Mercedes-Benz, for example, has an Attention Assist system that detects driver status based on driver driving behavior and based on vehicle state parameters (vehicle speed, engine speed, swing angular velocity, lateral acceleration, steering wheel angular velocity and angular acceleration).
If you think about it roughly, yes, the last time a drunk friend drove, as soon as you got into the car, you felt that something was wrong, which was judged by the acceleration and deceleration of the vehicle and the driving trajectory.
This minimalist approach, while effective, is sometimes unreliable. So let's compromise: camera + steering wheel sensor! This is also the current mainstream scheme!
Second, the perception algorithm: the driver's distraction, fatigue, drowsiness, coma...
The camera sees the face and eyeballs, how to judge the driver's state?
Moreover, the judgment cannot be whether it is OK or not, but it is necessary to distinguish between: distraction, fatigue, drowsiness, and even coma (sudden accidents, sudden illnesses)...
The traditional method is to artificially summarize a set of judgment rules, such as a review:
According to the frequency of the occurrence of tiny nods in 1998: when too large it is judged to be drowsiness. According to the proportion of 70% of the time the eyes are closed in 1994: when the closure is more, it is judged to be drowsiness. But this method is ineffective for drivers with small eyes and sunglasses.
Comprehensive PERCLOS Indicator, 1998: Blink duration, blink frequency, pronounced head or body movements. Monitor pupillary changes: the pupil shrinks during drowsiness and remains relatively stable when awake. If the co-pilot sits on the person you love, the pupils will also expand and small, thus interfering with the algorithm. Monitor changes in the mouth: frequent talking, serious driving, occasional yawning, mouth movements are significantly different.
There are many more of these algorithms, and I guess you don't want to listen to them anymore, in fact, there is no need to listen to them. Because......
This is because after the deep learning algorithm came out, it said: Everyone sitting here is garbage! 、
AI algorithms such as so-called deep learning algorithms do not summarize these rules at all, but directly give conclusions. It's like after Alpha Go's Go finished torturing human players, Ke Jie said to Alpha Go in a grievance: You don't know Go at all!
Alpha Go said to Kofa: You don't understand AI at all. Does it matter if you understand Go? I can win anyway.
In short, the deep learning algorithm is the following layered, one-way operation, you can judge the driver state, better than all the rules previously summarized by humans:
The beauty in ai's eyes, she looks like this.
You can say to AI: You don't understand beauty. Shouldn't a good beauty be seen according to the standards of three courts and five eyes?
AI tells you: You don't understand AI. What do I think of beautiful women? I knew when she was dozing off!
The main judgment is based on the camera, which actually integrates various states such as vehicle status, steering wheel sensor, microphone, etc. In the future, with the integration of automobiles and smart wearable devices, DMS will become more and more accurate and safer.
For example, when measuring ECG data, you also have to paste a sensor on the seat. Today, both the Huawei watch and the iwatch have ECG capabilities. If everything is available, they will be sent to build a car!
All sensor information goes through a data fusion process:
Third, the actual effect of the current level
What are the actual results? As far as the current situation is concerned, it is quite accurate and sensitive... Whether it's mind-wandering (eyes still looking straight ahead) or turning your head to speak, it can be accurately identified. Even wearing sunglasses can't escape the eyes of AI algorithms...
The 100 points of smart driving points have been deducted from only 52 points...
Finally, will it interfere with my driving? I don't think its reminders are well-intentioned, accurate, and for the safety of my life.
I don't hate it, but I have to thank it.