Car stuff (public number: chedongxi)
Author | Hao Han
Edit | Juice
L2 has frequent accidents, what is the problem?
Last week, on a highway section in Ningbo City, when a Xiaopeng P7 was driving with the LCC function (single lane L2), the vehicle system did not recognize the stationary vehicle in front of it, and rear-ended the front car at a speed of 80km/h, resulting in the unfortunate death of one person in front of the car.
Previously, the models of car companies such as Weilai and Tesla have also had similar accidents many times.
After the accident, topics such as whether automatic driving (L2 level) is safe and how to blame the accident have once again stood on the cusp of public opinion.
What needs to be emphasized here is that according to the classification of automatic driving levels, the LCC function of Xiaopeng P7 belongs to L2 level automatic driving, and the main responsibility must lie in the driver not taking over the vehicle in time.
In addition to the driver's distraction, frequent accidents may also indicate that there are certain problems in the L2 level automatic driving system itself at this stage.
Summarizing the L2 accidents in recent years, the accidents are mainly static vehicles and obstacles in front of the rear-end collision. That is to say, even if there are cameras all over the body on the smart car, all kinds of radar are available, and there is still a risk of the perception system failure.
So why is this problem? Is there any way to make up for this shortcoming? After in-depth analysis of accident cases and exchanges with industry insiders, Che Dongxi found possible answers.
First, the main responsibility lies with the driver, but the perception system also has to carry the pot
Judging from the surveillance video at the accident site, the Xiaopeng P7 did not significantly accelerate or decelerate before the collision, almost rear-ended the front car in a state of uniform speed.
At that time, the vehicle turned on Xiaopeng's LCC function, which means that adaptive cruise (ACC) and lane keeping (LKA) are turned on at the same time, which can also be understood as L2 automatic driving in one lane.
When this function is turned on, the vehicle will automatically control the speed according to the distance in front of the car and drive in the middle of the lane, and the owner can also customize the distance with the car.
The vehicle showed no signs of deceleration before the accident (leftmost lane)
Therefore, the vehicle does not have any deceleration or avoidance behavior before the collision, which proves to a certain extent that the vehicle is most likely not to perceive the obstacle in front of it or the perception result is wrong.
But this only shows that there is a certain problem with the vehicle perception system, and the cause of the whole accident is due to the driver's distraction and failure to take over the vehicle in time, and the responsibility lies with the driver.
The driver's distraction was the main cause of the accident
According to the owner's description, his Xiaopeng P7 is equipped with Xpilot 2.5 hardware, the perception hardware is composed of 1 monocular camera, 3 millimeter wave radar, 4 surround view cameras, 12 ultrasonic radars, and the computing platform is not the NVIDIA Xavier on the high-end model, but the Mobileye Eye Q4 chip with only 2.5TOPS.
Among them, monocular cameras and millimeter wave radar mainly provide perception for L2-level automatic driving, while 4 surround view and 12 ultrasonic radars mainly provide perception for parking.
Screenshot of the owner's readme
According to the data, the monocular camera of Xpilot 2.5 is basically the same as the mid-range perception camera of Xpilot 3.0, which is the same as 2M pixels, the horizontal FOV is 52 degrees, and the effective detection distance is between 30 and 70 meters. The millimeter-wave radar also uses Bosch's most advanced fifth-generation millimeter-wave radar, and the detection distance is also about 200 meters.
Camera configuration of Xiaopeng Xpilot
From the sensor configuration, before the accident, both the camera and the millimeter-wave radar should be able to detect the obstacle in front of them, and the vehicle can brake accordingly. But the actual situation is just the opposite, the owner of the car reflects that the system is not recognized, and the vehicle has no signs of braking.
This also makes people wonder, obviously the perception of hardware is there, why can't it be recognized?
Second, the vision of the special-shaped scene is not clear, and the radar filters stationary objects
For cameras and millimeter-wave radar, as long as it is normally working, it is absolutely possible to "see" the road ahead.
But the "seeing" here is limited to the visible, the root cause of the problem is that the visual recognition of the alien scene is not good, the millimeter-wave radar will filter the stationary object, and the car before the accident just met the two conditions of the "stationary alien car".
First of all, the visual perception system of automatic driving requires a lot of model training to continuously evolve, what scenes are trained more, the recognition success rate is relatively higher, and what scenes are trained less, the recognition success rate is relatively lower.
In the Xiaopeng L2 accident, the vehicle hit was an old Passat, and its body height and color were relatively close to the next barrier. Also, the vehicle was parked in the leftmost lane of the highway, and a white-clad passenger was crouching between the vehicle and the barrier, with an ice cream bucket on the right rear of the vehicle.
1 second before the accident
Obviously, the above scene is unfamiliar and complex for the visual perception system, so the "obstacle" in front of the car is difficult for the system to recognize as a vehicle that needs to react.
In addition, a few seconds before the accident, the owner of the car came to the rear of the car to adjust the position of the ice cream tube, blocking the wheels, tail lights and other components of the vehicle during walking.
Some insiders told the car that the visual recognition algorithm of the vehicle will rely on the wheels, taillights and other vehicle feature points for identification, and when the vehicle overlaps with the pedestrian, it will cause great interference to the recognition algorithm.
So, the inner monologue on the camera on the Xiaopeng P7 at that time was: "Is the front an obstacle?" Why have I never seen it? Why is it getting more and more blurred? Is there any? Let the millimeter wave radar confirm it for me."
Then, the recognized pressure comes to the millimeter-wave radar.
In simple terms, millimeter-wave radar is to transmit electromagnetic waves outward to generate echoes for ranging and speed measurement, and its perception does not rely on a large number of model training, so there will be no visual "entanglement" situation, see is to see, not to see is not to see.
Millimeter-wave radar on Tesla's older models
But millimeter-wave radar also has its own "bugs", which mainly rely on the Doppler effect to perceive moving targets. The characteristics of the Doppler effect are that dynamics are the easiest to perceive dynamics, dynamics are more difficult to perceive static, and static is extremely difficult to perceive static.
Therefore, if the vehicle in front is stationary, the target information is easily mixed with ground clutter, etc., and a certain algorithm is required to distinguish the target from it. If it is a moving car, based on its Doppler information, it is better to detect the target.
Millimeter-wave radar has a harder time sensing static objects
But at present, radar engineers such as Bosch, Continental and other companies have long solved the problem of identifying static objects from ground clutter, why can't they accurately identify stationary vehicles?
This is related to the current technological status of millimeter-wave radar - the general millimeter-wave radar has no altitude information and insufficient spatial resolution.
The lack of altitude information means that radar has a hard time distinguishing between road signs crossing the road and cars under the bridge; Insufficient spatial resolution means that two objects that are very close together, whose echoes are mixed together, are difficult to know how many targets there are.
If it is difficult to distinguish, and then the static target is mistakenly identified as a vehicle, and then the brake will seriously affect the user experience, and even increase accidents, so some radar companies and autonomous driving companies will choose to filter out static objects (including cars) to reduce the case of false triggering (ghost brakes).
Tesla appeared "ghostly" brakes
And the inner monologue of millimeter-wave radar may also be "There is indeed a stationary object in front of you!" How can there be a stationary car on the highway? It must be a bridge or a barrier! It's not a big problem, telling the visual front that everything is normal. ”
Moreover, many of the L2 automatic driving systems currently in mass production in China use vision as the main sensor (high weight), and radar as auxiliary sensor (low weight). If the visual spot is an obstacle, the vehicle will react whether the radar finds it or not, but not the other way around.
For example, if only the radar sees an obstacle in front of it, the system will wait for the visual result to give it before reacting. If the vision never gives the result, it simply does not react - after all, L2 still has a human driver pocket.
So specific to this accident, the possible reason is precisely because the vision did not see, or did not see in time, millimeter wave radar saw, but was filtered out, resulting in collision.
Third, lidar can enhance perception before fusion or as the optimal solution
In summary, the alien + stationary (low-speed) car or obstacle has become a major "BUG" in the L2 level automatic driving system at this stage.
Although the probability of this scene appearing is not large, most of it occurs in high-speed road conditions, and the casualty rate of accidents is relatively higher, so this "BUG" cannot be ignored.
In this regard, the simplest and roughest approach of car companies is to introduce lidar or even multiple high-beam lidar on the basis of existing cameras and millimeter-wave radar to greatly improve perception capabilities.
Compared with millimeter-wave radar and cameras, lidar has higher detection accuracy and higher density of point clouds. And lidar does not need to classify and identify the objects in front of it like the camera, only to confirm the existence of an obstacle in front of it through the reflected point cloud information, and then the system can avoid it, which can improve the driving safety of the vehicle in the face of special-shaped obstacles.
However, with the addition of lidar, the multi-sensor fusion scheme has added a new source of information, which cannot fundamentally solve the above problems, or there will be a problem of "fighting" between various perception results.
Previously, there were rumors that the confidence level of Xiaopeng P5's lidar in its perception system was not high, mainly based on visual perception. In this regard, Che Dongxi also conducted practical tests on the Xiaopeng LCC function after adding lidar.
The Xiaopeng P5 reminds the driver to take over after recognizing the stationary vehicle
At that time, there was a van in front of the lane chosen by the car, and then the ACC and LCC were turned on at the same time to ensure that the vehicle was moving forward under the assisted driving system.
Not long after departure, the Xiaopeng P5 recognized the stationary vehicle in front of it, and the vehicle dashboard clearly showed the presence of a car in front of it.
However, at this time, the Xiaopeng P5 did not significantly reduce the speed, but reminded the driver to take over on the car page, and the seat belt also began to tighten to remind the driver. In the absence of an active takeover of the car, the vehicle automatically exited the LCC, but still maintained ACC status.
This also proves that, at least at this stage, lidar is used, but it is not fully used.
Xiaopeng P5 front side lidar
So, how can we completely solve the problem of "fighting" each other between various sensors? What about learning from multiple sensors?
Specifically, at this stage, most of the vehicle's perception system is alone perception, independent processing, independent output, when all sensors complete the perception, and then by the system to carry out the fusion of all perception results. In this process, each sensor has different confidence levels in different scenarios.
Therefore, if the perception of each sensor is aligned at the pixel level, the system then uses the same perception algorithm to perceive the fused multidimensional data, and then output the perception result. This approach also does not have to consider which sensor's data to trust more, and does not have to struggle.
Eight cameras on the body converge into a three-dimensional "vector space"
In this way, the concept of confidence level no longer exists, and the sensors can learn from each other and integrate each other.
Of course, pre-fusion also has the problem of pixels being difficult to align.
Tesla has developed the "Vector Space" technology, which can draw a 3D bird's-eye view (BEV) based on the data input of 8 cameras, forming a 4D space and time label "road network" to present information such as roads, helping vehicles grasp the driving environment and find the optimal driving path more accurately.
In the BEV perception fusion algorithm, the ideal car integrates the information of lidar and high-precision maps, which further improves the perception ability under extreme conditions.
Therefore, at this stage, lidar is only the first step, and the pre-fusion between multiple sensors is the optimal solution.
Conclusion: Assisted driving systems are not autonomous driving
At present, Tesla, Xiaopeng, Weilai and other major new energy vehicle manufacturers have introduced advanced auxiliary driving functions, but the current autopilot systems require drivers to not leave the steering wheel with both hands and are ready to take over the steering wheel at any time.
However, at present, many consumers are using assisted driving as fully automatic driving, and dangerous scenarios such as driving to sleep and "driving unmanned" are emerging in an endless stream.
This accident has once again sounded the alarm for the use of automatic driving systems, car companies can not over-promote the ability of automatic driving to consumers, and need to force users to turn on advanced driving assistance functions, still need to pay attention to the road conditions in real time, to deal with the situation that the vehicle can not handle.