Produced | Tiger Sniff Technology Group
Author | Utada
The head map | provided by the interviewee, Wang Shiwei (middle) and the Tanwei Qinghua army, behind which is the upcoming shipment of lidar
At the beginning of 2022, Wei Weng, head of Velodyne's Asia Pacific region, quietly left his job.
The U.S. lidar giant, which dominated China's self-driving market for three years from 2016 to 2018, completely gave up the Chinese market after the board of directors was involved in two years of endless infighting and solid-state product research and development stagnated.
At the same time, most Chinese lidar manufacturers, such as Sagitar and Hesai, which have been on the stage of autonomous driving history due to the reverse dismantling of Velodyne products, have also come to a critical node where they must rely on their own strength and speed, rather than "reverse engineering ability" to survive.
On the one hand, the huge hype thrust of autonomous driving and Huawei's turmoil have made lidar the only choice for automakers to "fantasize about surpassing Tesla"; on the other hand, although the lidar used for obstacle avoidance and mapping has existed in the industrial world for more than 20 years, the solid-state lidar that can be embedded in the car needs to be redefined and built.
Mechanical lidar due to the complex system structure obviously can not pass the car regulations, pure solid-state route is not yet mature, mixed solid-state route has become the choice of most companies, but the cost has not yet fallen to the best expectations of the car manufacturer.
In other words, even in 2022, there is no fully qualified vehicle-mounted lidar.
In the state that all the technical routes have not run out, many startups have once again smelled new opportunities to swallow a huge cake: foreign countries, including dozens of solid-state companies such as Luminar and Ouster, which have been listed, are poised to go; domestically, in addition to the outstanding players such as DJI, Sagitar, Hesai, and Yijing, a team of Tsinghua Doctors who hold a number of solid state patents are also ready to join the melee.
Tsinghua Lab's "Stubbornness"
Back in 2017, visionary entrepreneurs sniffed out the business opportunity to break free from the narrow market of Robotaxi from the 4-line lidar produced by Valeo on the Audi A7. If you look back at the media reports of that year, the voices of Chinese entrepreneurs who expressed the "vision of mass production of solid-state lidar on the car" are endless.
It was at such a very tempting point in time that Wang Shiwei and 4 Tsinghua Jingyi brothers made the decision to set up a company - Tanwei Technology , "with 10 seconds of thinking time" at a post-year party in 2017.
Among them, Dr. Zheng Ruitong, CTO with more than ten years of experience in lidar development, is the pioneer of solid-state lidar and image fusion technology; there is chief engineer Zhang Zhengjie, who in addition to the double degree master's degree of Tsinghua University and RWTH Aachen University of Technology in Germany, also has a pair of cutting metal "golden hands", and once served as the technical director of CRRC Erqi Equipment Co., Ltd.
Wang Shiwei, who was pushed to the CEO position, as an optical expert who has participated in the major national secret satellite project, really has a comprehensive understanding of the automotive supply chain after joining the Academy of Information and Communications Technology in 2016 to participate in the formulation of auto parts standards.
"This tide is coming, and no one will be indifferent. Before the establishment of Tanwei, we had been working for several years, and we had been crushed by technology and the market for several rounds, and many things had been fully considered. ”
They feel that after more than a decade of tinkering with radar and optical instruments in the laboratory and the ultra-clean room, and being familiar with related technologies, they will be able to win tens of millions of orders from the depot in the future like Valeo.
Over the next two years, however, things took a sharp turn for the worse – there was no mature mass production technology, much less a market.
"Before Huawei explicitly said that it would go to solid-state lidar, in fact, the entire industry was in a wait-and-see state. At that time, mechanical radar 'dominated' the self-driving market, and the solid-state route not only had a bunch of technical problems, but there was no market yet. ”
Wang Shiwei remembers that before 2019, high-end autonomous driving only needs mechanical products to meet, and the automobile industry is only "thunder and rain"--traditional car manufacturers are still stuck in the irony period of thinking about whether "Wei Xiaoli" can survive.
Therefore, when 2019, high-end autonomous driving enters the first cold winter, the situation of lidar companies with only one thigh to hold has become extremely difficult.
"The essence of this is still a chicken or egg." For two years, they had to make 16-line mixed solid-state products first, breaking into the non-auto industry that could boost sales revenue.
"Of course, the situation is not good, because the solid state was not just needed at that time, even if the test was only a small batch of shipments, the depot could not afford to do so." Worst-case times... It should be the end of 2019, if the latest financing does not arrive, we will not even be able to buy a plane ticket to the United States to participate in CES. Wang Shiwei only downplayed the situation at that time, and felt that there was no need to render any emotions when the entrepreneurship was not successful.
"Survival is definitely to survive first, but fortunately, several partners are 'axis', yes, we just don't change the route." If there is anything special about us, it may be that we have never changed the technical route, we just want to be solid. ”
This sentence does tell the technology trend and chaotic state of the entire industry at that time.
As a 6-year observer of the lidar industry, I have seen that most startups have almost explored mechanical, semi-solid and solid-state. However, in order to survive, we have to first replace the "mechanical radar in china".
Traditional industrial-grade lidar manufacturers, who are also known for their mechanical radars, proudly compete with Velodyne's 64-line mechanical products while copying the technical route of Valeo's only vehicle specification products in order to get on the train.
But not all entrepreneurs are as lucky as Wang Shiwei's team.
Around 2019, several lidar startups that had received financing announced that they were "shutting down"; and those "old-school" laser detector manufacturers who tried to enter the in-vehicle industry from the industrial and mapping industries did not succeed in running out and retreated back to the original track.
Some industry insiders stated to us a point of view that many companies have abandoned on-board research and development: "Many companies hear that Huawei is coming in, like a deflated leather ball, engineers also feel that technology is not played, the investment is too large, it is better to return to their comfort zone of expertise, continue to engage in industry and mapping." ”
The idea is interesting because another view held by a head company is just the opposite: "It is precisely Huawei that has come in, indicating that this direction and the market are particularly dramatic." To compete with the strong is to be like a 'man'. ”
But no one expected that in just two years, it would once again usher in the upheaval and outbreak of 2021.
Wang Shiwei recalled that after 2021, the contact with automakers has increased suddenly, and investors have also taken the initiative to find the door, and the latter's judgment is almost only one basis - this is a track closely related to the subversive changes in the automotive industry. In September 2021, they successfully completed more than 100 million yuan of A round financing, and no longer experienced the previous "finding money" ups and downs.
Everything got a lot more interesting.
Although no one has said what is necessarily related to car sales in the next 5 years, this part that has exploded because of the concept of automatic driving has become the only Tier2 manufacturer that can "directly talk" with the car factory boss together with the "chip".
The last round of "cleaning" directly led to the sudden reduction of this wave of on-board lidar competitors, after all, there are companies in the industry that "hard copy" the German lidar system manufacturer Pick products have been used for 5 years, and this hardware track cannot escape the development law of the semiconductor and automotive industries.
Nowadays, in China, companies with certain solid-state technology and mass production experience, in addition to Huawei and DJI, a slap can be counted; on the other hand, more than 10 foreign vehicle-mounted lidar manufacturers are deployed on the most difficult mixed-solid and pure solid-state tracks.
In this not long list of solid-state lidar friends, Wang Shiwei's team can finally have a place.
Solid-state technology battle
When it comes to on-board lidar, most people must think of the huge steel gyroscope on the driverless car, which is one of the typical forms of mechanical products. Given that automakers will never sell you a consumer-grade car with a "tumor on your head," their special requirements for vehicle-grade lidar radar become intuitive and well understood:
Volume, stability, mass production, cost.
Pony Zhixing's Robotaxi, lidar is usually mounted on the roof
But as mentioned at the beginning, because Velodyne is the originator of mechanical lidar, which makes the mechanical "from 0 to 1" part completed, there are Chinese latecomers rushing to gradually accumulate and improve the part above 1, and finally drag the market into the price war;
To break into the solid-state products of the automotive supply chain, it is necessary to innovate and subtract from the above extremely harsh 4 dimensions, and from the product point of view, everything is back to zero.
Therefore, the reality is that although the car factory has established a relatively mature lidar cognitive system due to "automatic driving", there has not been a unified inspection and application standard so far.
According to an industry person who is very familiar with lidar, before 2019, the boss of domestic car companies initially thought that lidar was very simple - "isn't it like adding a camera". But then it turned out that no one in the team below could figure out how to add this thing.
"To be precise, 2018-2019 is the learning period of the depot, when many domestic and foreign depots have established lidar project teams to recruit talents and study how to use lidar in the end, including 'where to install it'."
He believes that at that time, foreign car companies went relatively faster, after all, the earliest industry originated from Europe and the United States, Valeo's products were the first to pass the car regulations, Audi was the first to load the car, and GM and Ford were also in close contact with lidar companies at that time. However, judging from the product progress and concept popularity after 2019, foreign countries have indeed lost to China's speed.
On the one hand, due to multiple factors such as cost performance, domestic lidar is gradually replacing foreign products. For example, a well-known traditional car manufacturer is preparing to replace the brand line that originally used ibeo lidar with a domestic manufacturer's product;
On the other hand, after accumulating sufficient testing experience, the depot is no longer easily deceived by "those product parameters such as detection distance, field of view, resolution, and light interference".
For example, in the past, various lidar manufacturers will write the detection distance to 100 meters 200 meters 300 meters, the larger the better, but now, the depot is basically clear "this number can be measured under what kind of environmental conditions", what are the characteristics of the detection target, and how much difference in the effect of light interference during the day and night will be.
"A domestic radar manufacturer's products can detect objects with 10% reflectivity at 200m, which is already very good." Generally, 10% reflectivity is equivalent to black tires, and such low reflective objects are usually used as a minimum standard. Wang Shiwei believes that the current depots are not Xiaobai and are familiar with the "horizontal evaluation" of light cars.
"And the questions that the automakers ask us are usually simple and crude — how much cheaper can you get than XX?"
The LiDAR of the Xiaopeng P5 (front left) was taken at the Shanghai Auto Show in April 2021
However, it is the increasing weight of lidar in the supply chain that creates new problems.
Up to now, the on-board lidar that once shouted that it would be reduced to 500 yuan unit price due to scale, the cost is still as high as several thousand yuan, and the yield rate of the production process of some products is not high; from the perspective of stability, the so-called "passing over the car regulations" only plays a basic role in the set of guiding principles of the old system.
"When Audi used Scala lidar in 2017, what the car rules looked like was actually Audi's own decision. Of course, there will be an industry consensus on environmental stability, after all, cameras and millimeter-wave radars must be tested for environmental stability.
However, the detection rate, recognition effect, and even the degree of cooperation with various sensors such as cameras on the car, in fact, there is no unified performance evaluation standard in the industry. ”
In her communication with the car manufacturer, Wang Shiwei found that many customers already have a preliminary mature test system, or have done a lot of reference from first-tier suppliers, but they do lack standards.
Someone once revealed to us that in 2021, a new force is so-called on the mass production model of two lidars, and the lidar on the front of the car has almost no effect. In simple terms, it is used as "car body decoration". This awkward marketing configuration may be repeated on other models in the future.
In addition, some people say that although IBEO's mass-produced products are the same size as the iPhone in terms of volume, they are not as good as expected in terms of heat dissipation; while the MEMS-type products (which is one of the well-known mixed solid-state technology routes) have incomplete vehicle specification verification and need to be improved in stability.
In essence, these problems are that the on-board solid-state lidar products are not mature enough in four dimensions.
Lidar is classified according to the "scanning method"
In fact, we often refer to lidar as "mechanical, hybrid solid and solid", the main basis of the technical route is "whether the scanning module will move, how many moving parts."
This is also one of the biggest technical difficulties in the current lidar passing regulations.
In the long run, pure solid state must be the mainstream route (for example, chip-level product stability is definitely the strongest). But today, whether it's Flash or OPA, all solid-state technologies are immature — or expensive or have short detection distances. Foreign companies that develop this technology have no good news.
So far, the technical route has not lost who wins, and each has its own advantages. Image from Minsheng Securities
As a result, most companies looking to move into the automotive supply chain as quickly as possible have opted for the hybrid solid-state technology route.
In this range, many manufacturers choose MEMS or other dual-axis galvanometer technology solutions. However, in terms of detection distance and environmental stability, there is still a state that is difficult to achieve, and it is necessary to continue to achieve process breakthroughs.
In fact, some industry insiders have told us that among all optical scanning solutions, reflective single-axis scanning is the most classic, stable, and easiest way to pass the car rules, and it is also the most basic way. This is also why head manufacturers including Valeo and Hesai have chosen this form in the mixed solid-state range.
"But the single axis also has a big problem, that is, how to achieve 'multi-line' while controlling the volume, to achieve a 3D effect." Wang Shiwei pointed out that most of the traditional uniaxial galvanometer and rotor products are single-line radar (of course, there are 4-8 line products such as Valeo Scala), so it is necessary to do more innovation in other parts.
He reminds us that because the market is always focused on lidar scanning systems, it is precisely because of the three other important components of it that are ignored - transmit, receive and signal processing circuitry.
"Lidar is much more complex than we think, and the proportion of the scanning part in the lidar structure is actually not high." Lasers and receivers are undergoing technological changes that are closely related to cost and performance. ”
Wang Shiwei pointed out that even if their scanning module takes the single-axis micro-galvanometer route, after making "array" innovations for the transceiver module, in addition to making multi-line effects, the difficulty of optical installation and adjustment must be reduced to the minimum-
With a significant reduction in the number of components, assembly and debugging time takes several minutes to complete.
"One of our core technology breakthroughs is the use of arrayed integrated devices at the transceiver module layer, which directly ensures that the detection resolution is broken from the first generation of 16-wire products to 64-wire, all the way to the current 192-wire solid-state version." Since the details of the transceiver system involve technical patents, he can only disclose basic information.
"Simply put, it is to use a low-cost, more stable single axis to achieve a multi-line perception effect."
The trend of technological innovation at the level of the transceiver system, the picture comes from Minsheng Securities
Only now have we realized that although the products of every lidar company, in various reports, will be simply summarized as a certain form of scanning. But in fact, because lidar is composed of hundreds of optical and electronic devices, the complexity is extremely high, far from being represented by "solid and non-solid".
Even, each company will have great differences in optical design, signal processing, integration methods, transceiver device brands and price choices.
For example, in the design of transceiver systems, most of the lidar manufacturers will use APD technology (above), and now there is a trend towards the development of single-photon device SPAD, but the mass production chip and cost performance are still insufficient, and only a few domestic companies are doing related products.
Therefore, we are not rushing to such a market conclusion:
The first batch of lidar on the car, although it is a gimmick for the depots, the actual effect should be much lower than its claimed object detection and perception capabilities;
The on-board solid-state lidar controversy, in the solid-state products to mature, the cost of mixed solid-state products continues to decrease, the mass production capacity continues to climb in the 3 years, there will still be no obvious high and low points.
Hardware convergence is irreversible
Just like in the semiconductor world, the ultra-heterogeneous computing innovation of multiple XPU fusion has set off a fourth-generation computing revolution, in self-driving cars, since data has to shuttle and collide between new and old sensors, breaking the original modular independent operation mechanism, it will inevitably break the "barrier" between hardware and hardware.
As a result, the hardware convergence trend is also happening on consumer cars that are now capable of loading 10 million codes a day.
As early as 2008, when Wang Shiwei was busy making image and point cloud data fusion technology for a national satellite project in the Tsinghua Precision Instruments and Measurement Laboratory, he must not have imagined that a very similar technology would have such an important role in the autonomous driving and automotive industries more than a decade later.
Multi-sensor fusion. This is a key step in "establishing a connection channel" for a variety of sensors such as lidar, cameras, millimeter-wave radar, etc. in the car, and preprocessing information for the car brain.
If you use an analogy, this is the ability to "put parts into a language system before letting the car run itself."
In general, the sensors of a robotaxi are distributed
However, the reality is very bone-chilling. This advanced technology related to autonomous driving is far from being understood, digested and absorbed by the automotive industry.
"Most companies with autonomous driving businesses or teams, when doing sensor data fusion, use the post-fusion processing scheme - the camera generates image data, and the lidar generates its own point cloud data, which is perceived separately, and then handed over to the main processor for fusion."
An engineer familiar with fusion technology told us that the most common method is to connect the wires of various sensors to a board, and then calibrate and process.
"But there is a big problem with the accuracy of this method, and the sensors in different positions on the car are difficult to align in space and time." And once the image is misidentified, you have to check the lidar 'perception', if the latter gives a contrary judgment, then who should you choose? This matter will always have to be artificially intervened or pre-made rules, but it is inevitable that the wrong instructions will be issued. ”
He gave the example of a car manufacturer that stuffed two lidars on either side of the front. But it is obvious that the position and direction of the camera behind the car and the two lidar are completely different, the scanning angle is very different, and there will undoubtedly be a defect in accuracy through "post-fusion".
And "pre-fusion" can solve these problems.
The characteristic of this method is that the raw data of multiple sensors is first "rubbed" together, and then thrown in a "neural network cauldron" for perception training. It's like combining multiple pieces of hardware to form a "super sensor" – you can see not only images, but also infrared and point cloud data.
The most famous disseminator and practitioner of this technology in China is the former star high-end autonomous driving company Roadstar (unfortunately, this company was dissolved due to infighting), and its chief scientist, zhou Guang, who later founded Yuanrong Qixing, has detailed the precision advantages of "pre-fusion":
"Suppose you have a mobile phone in your hand, and the lidar can only see one corner of the phone, the camera can only see the second corner, and the millimeter-wave radar can see the third corner. If a post-fusion algorithm is used, since only a portion of each sensor can be seen, it is very likely that the object will not be recognized and will eventually be filtered out. But in the pre-fusion, because it collects all the data, it is equivalent to seeing the three corners of the phone, which is very easy for the pre-fusion to be able to identify that this is a mobile phone. ”
Dimension exploration "blends" lidar and cameras together in a special way
However, the pre-converged solution encountered many technical difficulties in its development. For example, develop a perception algorithm that is tried to match the pre-fused data. For automotive sensor systems, it is almost unimaginable to achieve a fusion accuracy of 3 to 5 centimeters across 100 meters through system calibration.
Wang Shiwei also highly admires "pre-integration", a technology that is crucial to the car's autonomous driving capabilities. However, in addition to the incomparable advantages of perception accuracy, after Tsinghua Lab has done a large-scale satellite pre-fusion project for the country, he believes that for the automotive industry, it is more important to ensure that the system "follows up without human intervention".
"Unlike cars, satellites have a harsh objective premise: once you launch, it's hard to fix some problems. This also requires that we must do it at that time, so that the product can achieve all automated and orderly work in an unmanned state. ”
He gave an interesting example, when the Hubble telescope in the United States was launched, scientists found that it had a mirror that was not very good, and it was very blurry to see things. But there is only one solution, that is, to re-send a satellite and install a compensation mirror.
"We were doing national projects at that time, and we couldn't all be like hubble telescopes, constantly sending new satellites to repair, so we had to ensure that satellites automatically processed information in an unmanned state." And that's another important reason why we adopted 'pre-fusion' technology in the first place – we had to let the system understand how to automatically match pixels to point cloud data." ”
Therefore, Wang Shiwei and the Tsinghua brothers, the satellite core technology accumulated in the laboratory that year, especially the hardware-level pre-fusion technology with lidar as the core, under the encouragement of this wave of automatic driving, did some commercial innovation corresponding to the pain points of automotive software.
Just like the data fusion for satellites in those years, the structural integration of the hardware layer for cameras and lidar first allows the two to achieve spatial alignment and time synchronization based on coaxial optical systems, eliminating the need for calibration steps.
In short, it is to establish a seamless "space-time synchronization cooperation mechanism" that allows the lidar to be on top when the camera exposes image defects; when the lidar appears sparse point clouds, it is compensated by the advantages of the camera.
"Lidar from the beginning of the 4 lines, 8 lines, and then to the 16 lines, 32 lines, 64 lines and 128 lines, is there possible that there will be 214 lines, 512 lines behind? Unlikely, because this is not only time-consuming, but also the manufacturer should consider the balance between the number of lines and the cost. ”
Wang Shiwei feels that the accumulation of lidar technology is long, otherwise no one in the industry would have copied foreign radar for several years. It is better to take a different approach:
"We think back to some of the details of the project when we first did it, and we actually considered this problem. For example, it was found that the laser light on some tiny objects is very sparse. Then in the pre-fusion state, the perception system will 'spontaneously' rely on the camera for recognition, at this time, the resolution of the lidar is greatly reduced, which is actually no problem. ”
But he argues that this is not an excuse for the low line count of lidar, but "if you can achieve the effect of a 2k HD image with a 128-line laser, then why not do it?" ”
Frankly speaking, although domestic automakers are talking about "software-defined cars" today, most of them are far from having outstanding software strength.
In addition, despite their initial understanding of lidar, they still lack the ability to systematically apply multiple sensors (which is why they have long been led by the nose of Tier1), not to mention the pre-fusion technology involving multiple high-end sensors.
"The new forces will move relatively fast, and many car manufacturers are also interested in the former integration." Wang Shiwei, who has been in close contact with the depot recently, has felt the drastic changes from the depot side and more and more software layer pain points.
"We found that customers actually feel very strongly about 'data fusion' without calibration. Some engineers in the depot complained that the accuracy of space matching was not enough, especially in high-speed scenes, there were many mistakes, so there were many appeals. ”
Write at the end
When I asked Wang Shiwei whether there was a most celebratable victory node in the more than 4 years of ups and downs of LiDAR, he racked his brains but finally did not give an answer. Because he feels that the company is far from reaching the so-called "successful" level, the task in 2022 is actually more arduous.
Lidar manufacturers with solid-state samples need to cross the second mountain – either investing in factories or looking for cooperative foundries to expand production and sales capabilities as soon as possible. This will be the main problem that Wang Shiwei and his technical team will solve in the next few years.
"Frankly speaking, this is not our advantage, because the volume is not large now, and if the production capacity expands rapidly, we will also encounter a series of production challenges such as factory inspection, yield, delivery and so on."
But even so, their office in Haidian Dongsheng Science and Technology Park has been full of people in just half a year, and the faces of young engineers from famous schools such as Tsinghua and autonomous driving companies, perhaps representing the industry behind a technology company, are in a period of rapid rise.
In 6 years, some people have quit, some people have come in, some people have become unicorns, some people are about to go public. But we would love to see more tech entrepreneurs step into this unknown, competitor-sought-after automotive vertical.
Because here is not only the market supported by tens of billions of dollars, but also the most difficult automotive hardware innovation and the most powerful opponents.