Car stuff (public number: chedongxi)
The author | Wooden rice
Editor| Xiaohan
Choosing a pure visual autopilot route, Tesla did it wrong?
According to the Washington Post, tesla's complaints about the "ghost brake" problem surged to 107 in the past three months. In the first 22 months, there were only 34 cases.
The so-called ghost brakes mean that when the user turns on Autopilot or uses L2 automatic driving systems such as FSD, the vehicle will suddenly and without warning in some seemingly non-braking situation.
The car thing team has purchased a number of Tesla models for daily use, and we have encountered the phenomenon of ghost braking many times (although it is many times, it is still a small probability problem in the total use time).
When it happens, it is often a sudden sharp brake, heavy braking, so the experience for the people in the car is very bad, even can be said to be terrifying. At the same time, it is also easy to induce the safety hazard of rear-end rear-end.
After the surge in complaints, U.S. regulators have launched an investigation into Tesla's issue.
Statistics show that the increase in this wave of complaints occurred after Tesla announced the cancellation of millimeter-wave radar in autopilot systems – indicating that the pure visual route is directly related to the increase in ghost brakes.
▲ Tesla owner complaint data counted by the Washington Post
So what happened to the recent spike in ghost brake complaints? Did models that used to use multiple sensors have similar problems? Are multi-sensor fusion solutions really better than pure vision solutions?
First, a number of car owners suffered ghost brakes Regulatory agencies intervened in the investigation
At present, many car owners have complained to regulators that the ghost braking of Tesla vehicles has put them in danger of being rear-ended.
One owner said in a note: "My wife asked me not to use cruise control or Autopilot in the car because we had encountered Tesla's ghost brakes on road trips before, when the stomach that caused my wife to become pregnant was pressed. ”
Another owner wrote: "Sometimes there is nothing in front of my car and there is an emergency brake. ”
▲ Tesla owners encounter ghost brakes while driving
One owner of the 2022 Tesla Model Y said he was driving on Taylors And Pine Street in San Francisco when his car found a plastic bag a few feet ahead that didn't affect his driving, but the vehicle also underwent emergency braking.
Ben Morris, another owner of the 2021 Tesla Model Y, was one of those who complained to NHTSA. He said it was his third Tesla, but the car's ghostly braking problems were more pronounced than any other car he had before.
He wrote in an email that "while my 2017 Model X had ghost brakes before, it's very rare. But after the release of the vision-based autonomous driving system in May 2021, it seems that this problem will occur almost every day. ”
Recalling the ghostly brakes his wife encountered while driving on the highway, he said, "It braked so hard that it slammed the child seats in our car into the front seats." ”
"Fortunately, our children were not in the car." Ben Morris recalls.
In response to Tesla's recent increasing complaints of brake failures, Lucia Sanchez, a spokeswoman for the National Highway Traffic Safety Administration (NHTSA), said authorities are in dialogue with Tesla about complaints about ghost braking issues.
Lucia Sanchez said: "NHTSA has started to pay attention to complaints about Tesla's ghost brakes and is reviewing Tesla through our risk assessment process. ”
The process includes discussions with Tesla, as well as reviewing other data sources, including early warning report data. If the data shows that there may be a risk, NHTSA will take immediate action. ”
Tesla disbanded its PR department in 2020, so it hasn't responded to requests for comment from foreign media at this time.
Tesla has argued in the past that its autopilot suite of assisted driving features is safer than ordinary human driving when comparing collision data, foreign media said. But Tesla now faces increasing scrutiny from regulators, including recalls and safety investigations, which raise questions about the reliability of its assisted driving.
Notably, most of the recent complaints about Tesla Vehicles have been related to ghost braking issues — since November, of the 189 complaints about the Tesla Model Y and Model 3 from 2020 to 2022 and the Tesla Model 3 in 2019, 107 were about ghost brakes, a figure that accounted for 57%.
Second, the increase in complaints after removing the radar Experts: Pure vision should be carried by the pot
Last October, Tesla said that the failure of its ghost brakes was triggered by an FSD software update. As a result, Tesla recalled the FSD software version that was updated at the time. But after the recall, the complaint against the ghost brake did not drop but rose.
Interestingly, the timing of these owners' complaints comes just after Tesla stopped using millimeter-wave radar as a sensor for its autopilot system.
Tesla announced last year that starting in May 2021, it would stop equipping North American-made Tesla Model Y and Model 3 vehicles with radar and plans to build a vision-based autonomous driving system.
▲ Tesla official website statement
Tesla motors are equipped with eight surround cameras, which Tesla says provide "360-degree vehicle visibility at a range of up to 250 meters." In addition, it utilizes 12 ultrasonic sensors to detect objects around the vehicle.
The Tesla Model 3 is equipped with 12 cameras
After adopting this self-driving system based on vision sensors, some car owners complained to regulators that their cars seemed to perceive too "sensitively".
One foreign owner said that on one occasion his Tesla vehicle had taken emergency braking due to sensing a truck in the opposite lane — he said the vehicle was "almost parked in the middle of the lane."
Phil Koopman, a professor at Carnegie Mellon University who focuses on the safety of self-driving cars, said: "Ghost braking occurs when developers don't properly set decision thresholds to decide when there is an emergency and to distinguish between false alarms. ”
"What other companies have done is using multiple different sensors and cross-validating between them — not just multiple cameras, but also multiple types of sensors, such as radar and lidar, which are more complex sensors that can use lasers to map the dot matrix of the environment."
The professor said: "It is more difficult to determine the road condition with only one type of sensor because there is no cross-validation of different types of sensors. ”
Third, multi-sensor is easy to fight Tesla embarked on a pure visual route
Because vision and radar technologies have their own advantages and disadvantages, the existing L1~L2 autonomous driving solutions mostly use vision + millimeter wave sensor configurations.
First of all, vision, whether it is the use of traditional feature point recognition technology, or the use of deep learning technology, visual perception scheme has two major problems.
1. The phenomenon of missed inspection and false detection often occurs.
For example, special-shaped cars, obstructed vehicles, and vehicles in the wrong position are difficult to identify because there has been no targeted training, or there is less training data.
2. There is a problem that it is difficult to distinguish between true and false.
In special cases such as a huge photograph of a person at the rear of the bus in front of you, the visual system can easily identify this photo as someone in front of you.
Speaking of millimeter-wave radar, because it relies on echoes to perceive objects, it is relatively difficult to have true and false problems - there are echoes when there are objects in front, and there is no echo without objects.
But because there will be "noise" in the echo, such as two vehicles that are very close together, or between a stationary vehicle and a fixed object, the echo is doped together. Under the existing radar technology conditions, it is difficult to distinguish them precisely.
Vision and radar have their own advantages and disadvantages, so in the past 20 years, the L1~L2 automatic driving system of mass production vehicles has often combined vision and millimeter wave radar, as well as ultrasonic radar, for cross-verification to improve the accuracy of perception.
The multi-sensor solution may seem perfect, but there are actually problems. At present, most manufacturers use post-fusion (target fusion) to do the fusion of vision and millimeter-wave radar - each sensor is detected separately, and then the results are compared. So once the results of the two sensors are inconsistent, it is difficult to deal with.
The common practice in the industry now is to set the confidence level, that is, to choose a sensor that is more trusted under certain conditions, so as to solve the problem of the result fight. But in fact, this approach does not fundamentally solve the problem - whether it is choosing vision or radar, there are still the perception drawbacks of a single sensor.
Since the choice can not avoid mistakes, Tesla, which has strong AI capabilities, simply threw away the millimeter-wave radar and came up with a pure visual solution.
Andrej Karpathy, Tesla's senior director of AI, gave several examples of better pure visual performance in a public speech last year.
The first example is when the vehicle in front brakes quickly, the visual sensor and the millimeter-wave radar behave very differently.
▲When the car in front brakes urgently, the perception effect of the two sensors is compared
The yellow lines represent the distance, velocity, and acceleration images (from top to bottom) perceived by millimeter-wave radar, and the blue lines represent the perception results of pure vision sensors. It can be found that millimeter-wave radar has many occasions in which the distance is suddenly reduced to 0, the speed is suddenly increased, and the acceleration is suddenly 0. This is because after the sudden deceleration, the millimeter-wave radar does not track the vehicle in front of it very well, so it is restarted many times, just like the vehicle disappears repeatedly in a short period of time, and it appears 6 times, which is likely to mislead the automatic driving system.
The information perceived by the pure vision sensor and the information of the millimeter-wave radar roughly coincide, but there is no sudden change in distance, speed, and acceleration, and the performance is very stable.
The second case is an overpass common on the road.
▲The vehicle perceives the bridge as a stationary object and takes braking measures
Since millimeter-wave radar does not have vertical resolution, an overpass is considered to be an object stationary in front of it. After the sensor perceives the data fusion, the vehicle believes that there is a stationary object in front of the driving and judges the emergency brake. This situation is very dangerous on the highway.
The third case is the appearance of a large truck parked on the side of the road in front of the road.
▲Millimeter wave radar perceives distance is not as good as camera
At this point, millimeter-wave radar cannot tell whether it is a car or an ordinary stationary object in front of it. So millimeter-wave radar gave the task to vision sensors, which eventually sensed the truck in front of them at a distance of 110 meters from the vehicle.
If you use a pure visual sensor, you can sense the truck in front of you at a distance of 180 meters, and have clear perceptual information from 145 meters, and start to slow down.
The example given by Andrej Karpathy can indeed prove that the pure visual route is excellent, but judging from the current ghost brake complaints, the pure visual solution is far from perfect, and it also has its own drawbacks.
Fourth, the multi-sensor route also has a ghost brake front fusion or a fundamental solution
Pure visual imperfection does not mean that we have to return to the old road of vision + millimeter wave in the past.
The car-to-thing team has purchased a number of Tesla models at its own expense, and in the daily driving day after year, we have also encountered the problem of Autopilot ghost braking many times, and there are two main scenes that are prone to appear.
First, when the vehicle in the next lane is too close to the car, the system has a small probability that it will mistakenly think that the next car will change lanes or add congestion, and it will directly come to a foot of heavy braking.
Second, when it is closer to the vehicle in front and the front car is frequently accelerated and decelerated, the car will be mistaken for the front brake stopped, and then it is a foot rebrand.
Because the ghost brakes are heavy braking almost every time, the experience is very scary and there is a risk of being rear-ended by the rear car.
In addition, once when driving at night, on the empty highway without cars, the car also encountered a ghost brake. It is possible that the overpass opposite or overhead was mistakenly identified as an obstacle and the brake action was taken.
The car thing believes that to better solve the ghost brake problem, it is not to go back to the old route, but to increase the lidar, and even supplement the imaging millimeter wave radar, using the method of pre-fusion, aligning the data of the sensor at the pixel level for perception, so as to truly play the advantages of multi-sensor fusion (of course, there is also a problem that pixels are difficult to align before fusion).
Musk, although diss lidar many times, thinks its price is too expensive. However, from the current actions of domestic lidar manufacturers such as DJI, Huawei, Sagitar Juchuang, Hesai Technology, and Yijing Technology, the cost of lidar is obviously becoming more and more not a problem.
After the cost of lidar has changed fundamentally, we still hope that Tesla can use it, create more high-quality experiences with more advanced sensors, and lead the industry to continue to move forward.