One of the most important sensors for autonomous vehicles is once again ushering in the golden age of the industry.
Author | Tetsu Ta
Edit | Wen Liang
In-car cameras have, to some extent, become the most important sensors in cars.
Whether it is Tesla's pure visual route or the multi-sensor fusion scheme commonly adopted by the industry, in-vehicle cameras are an indispensable member of it.
The rapid development of smart cars has also led to an increasing number of in-vehicle camera applications, and its market size has increased accordingly.
ICV Tank predicts that the global car camera market size will reach 27 billion US dollars in 2025, and the scale of China's car camera industry is expected to reach 23 billion yuan. Recently, canon and other companies that were not in the automotive field have entered the field of in-vehicle cameras, which has also promoted the development of this market.
Under the fierce competition in the industry, the new intelligent driving tries to explore what kind of development will be ushered in by the on-board camera in the era of automatic driving? What are the opportunities and challenges?
The past and present of the on-board camera
In 1956, camera technology first appeared in the Buick Centurion concept model in the form of a rear camera. After nearly 40 years of development, Toyota launched the world's first commercial car with a rear camera, the Soarer, in 1991, which became the beginning of the commercial use of in-vehicle cameras.
After 2006, the car camera industry ushered in a period of rapid development. Four scholars, K.Kate, M.Suzuki, Y.Fujita and Y.Hirama from Toshiba and other companies and institutions, jointly proposed the concept of panoramic surround view of the car - by installing multiple wide-angle cameras around the car body to collect peripheral images, forming a 360-degree panoramic view.
This concept attracted the attention of many automobile manufacturers, and in the following years, several OEMs, including Nissan and Fujitsu, introduced related systems. At the same time, the increase in the number of on-board cameras has also made cars more functional.
According to the different installation locations, the vehicle camera can be divided into front view camera, surround view camera, rear view camera, side view camera, and built-in camera.
The front view camera is mainly installed on the front windshield, which can be used to assist in the realization of front car collision warning, lane departure warning, traffic sign recognition, pedestrian collision warning and other functions.
Surround view cameras, which are mainly installed near the logo and integrated into the left and right rearview mirrors, are the basis for panoramic parking.
The rearview camera is mainly installed in the rear trunk and can assist parking by displaying a reversing image.
The side view camera is mainly installed under the rearview mirror for blind spot monitoring.
Most of the built-in cameras are installed in the rearview mirrors of the car to monitor the driver's status and realize functions such as fatigue reminders.
According to the data of the Prospective Industry Research Institute, the market penetration rate of the rearview camera and the frontview camera in 2019-2020 is as high as 50% and 30% respectively, and the market penetration rate of the side view camera and the built-in camera is relatively low, 22% and 7% respectively.
In recent years, the development of smart cars has given in-car cameras more functions than safety.
At present, the cameras of some smart cars can complete identity authentication through face recognition to unlock the vehicle and start, face payment, seat adjustment and other functions. With the development of smart cockpits, in-car cameras can also provide personalized services for the preferences of drivers in the car through face recognition and gesture recognition.
The emergence of new car-making forces has made on-board cameras increase social functions.
For example, models such as Xiaopeng G3 and Zhiji Automobile are equipped with 360° panoramic cameras on the roof, and users can control the lifting and rotating of the cameras through mobile phones and cars to capture the scenery during the driving process of the car, and support the forwarding of content to social platforms. In addition, the Xiaopeng P5 also supports interconnection with DJI drones.
After decades of development, the segment of today's car camera industry has formed a relatively stable market pattern.
Overall, at present, the world's well-known Tier 1 companies account for a relatively high market share of global car cameras, and Japanese and Korean companies also have a high market share in some segments.
However, in some segments, the global market share of Chinese manufacturers is gradually expanding. The field of filters such as crystal optoelectronics, kinette, gluing materials such as Hengcheng Weiye and so on.
Especially in the lens group supply chain, Shunyu Optics, which is deeply rooted in the optical track, is one of the players that cannot be ignored. According to its 2021 data, its in-vehicle lens market share has been the world's first for many consecutive years since 2016.
In-vehicle cameras in the era of autonomous driving
In 2017, the mainland government issued the "Medium- and Long-term Development Plan for the Automotive Industry", which pointed out that by 2025, highly and fully autonomous vehicles will begin to enter the market.
The guidance of policies and the increase in market demand have attracted many domestic companies to enter the field of on-board cameras and manufacture new on-board cameras to meet the needs of autonomous driving development. At present, dozens of domestic companies, including Hikvision, Huawei, DJI, And Forretec, have a variety of layouts in different types of vehicle cameras.
(Source: Shui Qing Mu Hua Research Center)
Regarding the camera scheme used in automatic driving, it can be roughly divided into two types.
The mainstream view is that if driverless is achieved, the bicycle must be equipped with five types of cameras, and each type needs 2-3 cameras with different focal lengths. Level 5 autonomous driving is expected to require 12 to 15 cameras.
Mobileye, known for its cameras for autonomous driving, is trying to reduce the use of cameras while introducing a pure camera self-driving solution.
At this year's CES conference, Mobileye once again said that it is building an autonomous driving system based on millimeter-wave radar and lidar. It is expected that by 2025, it will be equipped with a lidar on the front of the car, and with a 360-degree millimeter-wave radar to achieve automatic driving. It says the addition of millimeter-wave radar and lidar subsystems will improve the safety of cars and extend the mean time between failures.
In view of Mobileye's route, Li Lele, general manager of Desay SV Intelligent Driving Assistance Business Unit, believes that due to the physical limitations of various sensors, it is unlikely to use only one sensor to solve the perception reliability of all scenarios in the short and medium term, and if smart cars need to have highly reliable environmental perception capabilities, they must need a variety of sensor fusion to improve perception performance.
Therefore, in the development of autonomous driving, cameras, millimeter-wave radar, lidar, ultrasonic sensors, etc. are indispensable, and even V2X ultra-vision sensors need to be deployed.
Li Lele said that self-driving cars will be equipped with more than 10 cameras, 5 or more millimeter wave radars, 12 ultrasonic sensors, 1-6 lidar (different configurations according to the scene and function) and 5G+V2X super vision sensors. He predicted that the super vision sensor will be gradually listed in the next two years.
Large-scale mass production is still not yet coming
Autonomous driving requires an increase in the number of on-board cameras, but also requires a significant increase in its hardware performance.
At present, the maximum resolution of in-vehicle camera lens imaging is generally between 2 million pixels and 5 million pixels. Under the promotion of the driver assistance system, the 8-megapixel on-board camera began to enter the mass production model.
Last year, Weilai CEO Li Bin announced that the ET7 with NAD NIO's autonomous driving technology will be equipped with 8 million pixels of self-driving HD cameras, and Ideal Auto also revealed relevant plans last year.
Li Lele introduced that the 8-megapixel camera is mainly used for forward perception of intelligent driving solutions of L2+ and above.
Compared to the detection distance of the 1-megapixel camera is about 30 meters, the 8-megapixel camera can detect objects more than 200 meters in front of it because of its high resolution. In addition, the larger field of view of the 8-megapixel camera also makes the perception distance greatly increase while perceiving more refined content.
In addition to the development needs of autonomous driving, more factors are also driving the mass production of 8-megapixel cameras on the car.
First of all, thanks to the release of the large-scale automatic driving chip and mass production applications, higher-pixel on-board cameras are supported to collect image data, reducing the difficulty of processing data and the overall system cost of the computing platform. The NIO ET7 is equipped with four NVIDIA Orin chips, which can provide more than 1000 TOPS hashrate.
Second, the technology upgrade brings better performance to high-pixel cameras in low illumination conditions.
Although high-pixel cameras can provide longer and sharper images, compared to lower-pixel cameras, because the 8-megapixel camera has a smaller light-sensitive area per pixel on average, in the case of poor lighting conditions, it will make the image more noisy and blur the image.
At present, some manufacturers have optimized the CMOS image sensor for the key components that affect this part, and this problem has been successfully solved.
Although some production models have adopted 8-megapixel cameras, it does not mean that they will usher in the opportunity for mass production. It is reported that the cost of on-board cameras suitable for high-grade autonomous vehicles is as high as 400 yuan to 600 yuan, which is not conducive to the promotion and application of high-pixel cameras to a certain extent.
According to Li Lele's analysis, due to the shortage of semiconductor production capacity, the cost of chips such as CMOS image sensors and serializers/deserializers remains high, resulting in a corresponding increase in the cost of high-definition cameras. He expects that in the next two to three years, with the cost of chip production capacity increase, the configuration rate of on-board cameras will increase accordingly.
Huang Huan, chairman and CEO of Jinghong Technology, who has a layout in the field of car cameras, believes that compared with mobile phone cameras, the demand for high-definition cameras in cars is less, which leads to the gross profit margin of vehicle cameras as high as 30%-35%. When the market demand is large enough and the production capacity of in-vehicle cameras is increased, its cost will be reduced accordingly.
Huang Huan predicts that the development of car cameras will be the same as the development process of mobile phone cameras, and the lens pixels will be increased to 10 million, 12 million pixels and above.
The knockout rounds begin
In the future, the competition of vehicle-grade cameras will further intensify.
The research and development cycle of on-board cameras is generally 1-2 years, and if it needs to pass the vehicle specification level verification, its cycle will be as long as 1-2 years.
The industry expects that because of the long vehicle specification verification cycle and the higher requirements of the demand side for the safety and stability of product quality, the competition in the in-vehicle camera industry will intensify this year, when there will be leaders and laggards in this track.
Li Lele agrees with this view, while adding that market competition will also be intensified by two factors.
First of all, from a long-term point of view, due to the higher requirements of the algorithm for the camera, the strong coupling relationship between the domain controller and the data transmission between the domain controller and the camera, etc., will affect the choice of OEM, and the competitiveness of suppliers who only produce and manufacture cameras will decline in the future.
Secondly, because of the high industry market share such as intelligent cockpit and intelligent driving domain controller, especially in the current semiconductor shortage, if the manufacturer produces cameras and domain controllers at the same time, it will be easier to obtain the guarantee support of partners, which will be a short-term factor that gradually opens up the gap with competitors.
The intensification of competition in the car camera industry can be regarded as an industry pattern or will usher in a reshuffle on the one hand, on the other hand, it also means that the industry will show a rare window period, especially for Chinese companies in related fields, which may be an opportunity to go to the world stage.
The golden age of the car camera industry is accelerating.
END
Alternative lidar? The "Heat" and "Pain" of 4D Imaging Millimeter Wave Radar
Intelligent driving of the new war: the rush to "lightning fusion", five forces separately encircled and suppressed