After a hundred years, automotive glass rearview mirrors have undergone earth-shaking changes, from the earliest dressing, to the later safe driving essentials, and then to today's highly intelligent electronic. The advent of electronic rearview mirrors (CMS) has made rearview mirrors brilliant again.
"Gaogong Intelligent Car" learned that because the electronic rearview mirror has the advantages of large field of view, small blind area, small weather interference, etc., more and more car companies including Audi, Mercedes-Benz, Lexus, WM, General Motors, Hyundai and so on have released new models equipped with electronic rearview mirrors, and it will be the general trend to replace traditional rearview mirrors in the future.
Relevant information shows that the national mandatory safety standard GB15084 is expected to be officially released this year and implemented on January 1 next year, when the traditional exterior mirrors will be cancelled and new models loaded with CMS will be allowed to be sold.
This means that the door to a new CMS frontal packaging market is about to open. According to relevant data, the total annual global car sales are about 80 million, and the promotion of CMS regulations will directly spawn new tracks with a scale of 100 billion.
At present, many parts suppliers on the market are evaluating traditional SoC chips, such as IVI central control display chips, DVR recorder devices and even consumer security monitoring chips to achieve CMS design.
However, according to the effects of several models currently equipped with CMS and have been mass-produced, the performance of traditional SoC chip solutions in cmS systems seems to be somewhat "unsatisfactory".
For example, the AUDI e-Tron equipped with the Ficosa electronic exterior mirror scheme generally feels color cast throughout the picture, especially under the light pollution of reversing lights and brake lights, there are problems such as large dark noise, more masses and ghosts, and poor fluency. The full range of Internet celebrities with CMS standard Honda e also exposed problems such as large delays, non-smoothness and serious color casts in the image.
Image source: Ficosa official website homepage
Image source: foreign video website
Should the CMS system adopt a smart cockpit approach?
In the early days, most people's understanding of electronic rearview mirrors (CMS) still stayed in the reversing rearview mirror, streaming media rearview mirror (installed above the driver in the car) and other imaging products, thinking that the electronic rearview mirror (CMS) is a simple display + camera composition of the vehicle application.
However, combined with the experience of successful mass production of CMS overseas in the past, it can be clearly seen that CMS is completely different from the traditional understanding of the idea of pure camera and display. Limited by the regulatory requirements of various countries and the constraints of various models of interior and exterior decoration regulations and experimental standards, there is no traditional SoC solution on the market that can meet the CMS system requirements of various models.
At the same time, due to the huge differences in regulatory requirements and functional safety levels across countries, smart cockpits or autonomous driving domain controllers are compatible with replacing CMSs, which does not seem to work. CMS under the E/E architecture will be the mainstream form at this stage.
In fact, compared with the above-mentioned problems such as image color casts, the overall latency indicator of CMS is more worthy of attention. The reason behind this is that the scenery people see from traditional glass mirrors is the speed of light, but if a digital camera system is used, the delay control of the entire system will face huge challenges. Latency not only affects the user's simple cognition of "fluency", but more importantly, latency also involves security.
At present, although UN-R46 and GB15084 stipulate the lower limit of system latency, in the fields of communication, semiconductors, artificial intelligence, data processing, automatic driving and other technologies, low time latency is the goal of developers to improve the user experience.
1. "Sense of Fluency" under the subjective and objective indicators of the CMS system where "user experience" is paramount
For human vision, visual persistence characteristics determine the best frame rate for our image perception, generally speaking, humans do not feel the difference between 1,000 frames of pictures and 60 frames of pictures in less than 1 second, but the delay of tens of milliseconds or more will make most people feel uncomfortable.
Therefore, the industry generally believes that 60fps can be used as an optimal frame rate, and use this to determine the upper limit of "input delay". If the frame count of the monitor is 1fps (that is, one frame per second), it means that the screen needs to react to the input in between 0-1 seconds. If you enter just before the screen refreshes, the screen may react immediately, and once you enter after the refresh, you need to wait 1 second to see the action on the screen, which causes the input delay. Calculated at the optimal frame rate of 60fps for human image perception, the upper limit of input latency is 16.667 milliseconds.
The so-called "input delay" refers to the delay of a series of processes such as the subject image from the camera to the ISP, image processing, and conversion to the format supported by the display, and finally to the screen driver chip, waiting for refresh. Due to the extreme complexity of this processing process, the entire CMS system cannot achieve the speed of light like an optical mirror.
So, how much delay does the CMS system need to be so that the driver can't feel the delay? Some insiders said that everyone's senses are different, but theoretically, the first pixel of the camera's image sensor can correspond to the first pixel of the display, and the delay of the entire system can be infinitely close to the speed of light.
However, in the real process, CMS can not achieve peer-to-peer synchronization at all, especially based on traditional SoC schemes. The reason behind this is that the power-up timing and requirements of each module of the traditional SoC are different, and the synchronization between modules requires caching and the intervention of the operating system, resulting in an extremely complex and interdependent time synchronization relationship.
Therefore, it is difficult for non-dedicated, targeted design SoC chips to control the overall latency of the CMS system within 80ms. If the overall latency of the CMS system is to be reduced, only a strong parallel processing capability and a highly flexible FPGA can meet the requirements.
In summary, according to the barrel theory, considering the response time of the liquid crystal and the upper limit of the delay of the entire system should not exceed 40ms, it is also in line with the PAL-made film 25fps, 40ms per frame of the public simple cognition. If "user experience" is a big mountain of CMS popularization, then the extremely low latency is the tunnel through this mountain, and the faster it passes through, the more capable it is to run to the first. According to another source, the revision of the regulations has begun to consider the inclusion of customer complaints caused by the CMS experience, and subsequent automakers will "recall" this.
2. The system stability and durability requirements are extremely high. Unlike traditional car rearview mirrors, electronic rearview mirrors have to work all the time after the driver gets in the car. Once the electronic rearview mirror system has a frozen screen and a camera failure, it cannot effectively display the blind spot road surface, which will be a disastrous result for the driver's parallel decision-making, which also puts forward high requirements for the stability and durability of the entire CMS system.
3. CMS products as necessary safety parts for intelligent networked vehicles, the complexity of the scene will put forward higher requirements for the hardware statistics and algorithm processing of ISP signal processors, such as colorful body and rear headlights. Its entire color system and AE stability processing, higher performance HDR (up120db) requirements, LED lamp flicker suppression, etc. are facing new challenges, in fact, many mainstream ISPs can not solve the existing known problems, as mentioned at the beginning of the article.
At the same time, the subjective image index objectification of CMS is also a difficult point, image quality evaluation and control of indicators need to rely on expensive equipment and professional team support, CMS image debugging not only involves the image quality (IQ) of the ISP, but also synchronously adjusts the display quality (PQ) of the display. At present, there are not many domestic enterprises with these two professional capabilities and related equipment, as far as the image is concerned, most of them at home and abroad are still in the state of relying on the experience of debugging engineers as the "ceiling" of user experience. This is also one of the core pain points in the field of automotive imaging.
The flexibility of ISPs is extremely important in the CMS space. Compared with the traditional ISP or SoC, under the demanding indicator requirements, the IQ optimization ability of the traditional SoC or ISP and its image debugging service team is the "upper limit" of the product, and for the FPGA-based ISP, IQ optimization is just the beginning, the ISP hardware pipeline and core algorithm can also be adjusted or reconstructed, all of which not only does not affect the development cycle of the product, but can improve the core competitiveness.
According to a number of industry insiders and relevant experts of the main engine factory, in the specific environment where the regulatory standards are still being updated, the FPGA solution with the advantages of extremely low latency, hardware upgradeability, and flexible adaptation is undoubtedly the best choice for CMS systems. Fast on-board Demo and fast compliance review after the implementation of regulations, so that there is no need to change the original design, FPGA this feature is undoubtedly the most anticipated solution for cmS companies today.
At this stage, including the head suppliers of mature mass production such as Gentex and Mekron, the CMS systems they have delivered use FPGAs as their core image processing units, which to some extent also shows that the flexibility and reliability of FPGAs at this stage are trustworthy.
What is the best chip solution for CMS systems at present?
As we all know, the electronic rearview mirror (CMS) industry is still in the early stages of development, not only the regulatory standards have not yet landed, the hardware and software technology of the CMS system is also constantly upgrading.
For example, the LCD screen solution used in the CMS system on the market can easily lead to visual fatigue in the driver, and there are also certain problems in the clarity, field of view, and light imaging capabilities of the camera, and the follow-up will be specifically for the CMS system to upgrade and iterate.
Hua Sheng, automotive business development manager of AMD-Xilinx Greater China, said that the FPGA solution can meet the differentiated needs of different customers to the greatest extent, even if it is the replacement of core materials such as screens or cameras, only need to directly update the logic unit and core functions for hardware characteristics, thereby greatly reducing the development cycle and cost of CMS manufacturers and OEMs, and accelerating the mass production of CMS products.
It is understood that AMD-Xilinx and its partner iWaySense have jointly created a core image processor IP dedicated to CMS products, and have been specially designed for CMS application scenarios, with excellent image quality at the same time, but also can achieve extremely low latency. In addition, combined with the advanced image laboratory and debugging service guarantee capabilities of image debugging partner iWayLab, as well as the domestic localization team, it can further accelerate the start-up and mass production of the CMS project of the main engine factory.
Huasheng said that based on the iWayCam CMS series dedicated image processing IP solution provided by AMD-Xilinx image IP partner iWaySense, it can be deeply adapted according to the characteristics of the display and image sensor, so that both parties work in the best timing, so as to achieve extremely low latency and guarantee excellent performance.
At the same time, the flexible design can support the latest and most advanced sensors and screen display technology, and can achieve the ability to open the LED flicker suppression can also maintain the high dynamic range of 140db, based on the IP CMS machine boot-up map time is less than 1 second, the whole machine can do Power Down design, CMS standby dark current is very low, the whole machine function safety can reach ASIL-B, The system delay limit at room temperature can be ≤9.6ms (limited by the bottleneck displayed on the screen).
"Currently, there is no ASSP/ASIC device architecture on the market specific to CMS systems. At the same time, CMS technology is still in the process of continuous iteration, directly using the existing ADAS visual ASIC chip or fixed CPU, GPU solution, Demo may not be a big problem, but compliance mass production may still have a long way to go. Even if some integrated circuit manufacturers plan such dedicated ASSPs/ASICs, the cycle is estimated to take 3-5 years at this stage when the demand has not yet converged, from design to film to certification and mass production. So, there is no better fit for CMS systems than the AMD-Xilinx FPGA in the short term. The relevant person in charge of AMD-Xilinx added.
In fact, from the perspective of the demand side of the depot, at present, some depots will incorporate the CMS into the intelligent cockpit domain control, and some depots will be connected to the driving domain control to achieve a series of artificial intelligence functions. This means that it is also very likely that the image data of the CMS will be delivered to different "domains" in the future, and this demand must require a great deal of flexibility and compatibility with different protocols.
AMD-Xilinx's FPGAs and SoC devices support more than 90% of the world's physical and protocol layers, which has been fully demonstrated in more demanding communications fields such as 5G. At the same time, amD-Xilinx's Zynq MPSoC chip internal FPGA and IO resources can support the sensor access capabilities of multiple 4K@30fps, can flexibly bridge and adapt various interfaces, and can also implement image processing algorithms such as ISPs.
In addition, the Zynq MPSoC family of devices also includes a 4-core Cortex-A53 APU, Mail400 GPU, dual-core R5_RPU, and a PMU (Platform Performance Snap-in) with 3-core redundancy, which is fully capable of running AI algorithms independently, even without relying on peripheral computing power.
More importantly, using Zynq MPSoC can also greatly simplify the peripheral BOM, reduce nodes, further reduce costs and improve the reliability of the entire CMS. It is understood that the CMS scheme based on Zynq MPSOC automotive-grade devices can achieve the functional safety goal of ASIL-B level as a whole. The RPU consists of a dual-core Cortex-R5F, which can achieve ASIL-C level of functional safety under interlocking. The built-in PMU (platform measurement unit) has the initialization of the system before startup, as well as software test library execution (STL) and system error handling functions, can reach ASIL-C level, and can be accurate and reliable power management.
"Gaogong Intelligent Car" learned that the US Department of Transportation has launched a safety test evaluation of camera replacement exterior mirrors, and China is also expected to introduce relevant standards to allow the cancellation of traditional exterior mirrors and new models equipped with CMS.
In this context, major chip manufacturers, including Qualcomm and Weier, have begun to compete for CMS, a market depression. However, in the view of "Gaogong Intelligent Car", Whether it is from the perspective of extremely low latency, flexibility, safety and reliability, ecological cooperation, etc., AMD-Xilinx's FPGA solution has a great competitive advantage in the current CMS market.