News Highlights:
· The Mali-C78AE Image Signal Processor (ISP) is a new addition to Arm's "AE" product family with functional safety, the processor IP for advanced driver assistance systems (ADAS) and human vision applications
· The Mali-C78AE, paired with the Cortex-A78AE CPU and Mali-G78AE GPU, provides a superior ADAS image data processing pipeline
· Mobileye was the first to license the Mali-C78AE, and with the Mali-G78AE, it will be applied to the new generation of EyeQ technology
Arm announces the launch of the Arm Mali-C78AE ISP, adding a new addition to its IP portfolio designed for performance and safety in automotive applications. The Mali-C78AE, paired with the Cortex-A78AE and Mali-G78AE, provides a complete ADAS image data processing pipeline that optimizes performance, minimizes power consumption, and provides a consistent approach to functional safety to drive the next phase of ADAS production adoption.
Arm Mali-C78AE
According to a recent report by research firm Strategy Analytics, the total value of the automotive camera market is expected to grow by more than 19% from 2020 to 2025, making it the most important type of sensor that can provide vehicles with the data they need to make decisions about their surroundings.
Chet Babla, vice president of Arm's Automotive and IoT Business Unit, said: "As the number and precision of in-vehicle cameras increase, the computing power of efficiently and safely converting high-throughput image data into outputs that meet the different requirements of machine and human vision also needs to increase accordingly. The Mali-C78AE meets the industry's need for new methods of image processing, helping to enable new capabilities in ADAS and autonomous driving. ”
Human and machine vision require safety first
The Mali-C78AE is designed for human and machine vision safety applications required for a variety of ADAS functions and is capable of processing data from up to 4 live cameras or 16 virtual cameras.
The Mali-C78AE has been developed with hardware safety mechanisms and software diagnostics from the outset, enabling system designers to meet the requirements of ISO 26262 ASIL B functional safety. The Mali-C78AE is designed to prevent or detect faults in a single camera frame that can lead to frame data processing errors. To this end, the ISP has more than 380 fault detection circuits, continuous built-in self-test, and can detect sensor and hardware faults in the connected cameras.
Visual processing requires a lot of data and a high workload
Processing speed is a key element of the Mali-C78AE, no less important than security and user experience. For example, the entire process of acquiring an image from a sensor, processing it sequentially through an ISP and GPU, and then displaying it on a screen for the driver to see should be completed within 150 milliseconds; if the process takes more than 150 milliseconds, the driver will perceive a delay in the image when using functions such as parking assistance systems. In machine vision applications, the vehicle should not move a distance of more than 250 mm from the acquisition of camera images to the entire process of decision processing; if this distance is exceeded, it means that the machine vision system is too slow to meet the accurate and timely decision-making requirements required to cope with the driving environment.
To enable the driver and machine to make the best decisions, ADAS cameras must collect the most relevant information from each frame. The Mali-C78AE uses advanced noise reduction technology and dynamic range management to ensure that each frame is clear and exposed properly by adjusting areas that are too dark or too light. The Mali-C78AE is capable of processing up to four high-resolution, high-frame rate camera data in real time, significantly reducing memory, communication, and processing requirements, thereby increasing system efficiency.
By adjusting areas that are too dark or too light, the Mali-C78AE ensures that each frame is clear and exposed appropriately (Figure 2).
Currently, to implement multiple ADAS functions, a separate camera setup is required, as cameras used for machine vision applications, such as lane departure warnings, do not produce images that are appropriate for human vision, such as looking around. To reduce the cost of implementing multiple ADAS functions, the Mali-C78AE enables dual-use camera sensors by reducing the sensor output for machine vision and color conversion, creating images that fit the human eye. By avoiding repeated deployments of cameras and their associated electronics and harnesses, OEMs can save costs and reduce complexity, allowing for wider deployment of camera-based ADAS capabilities across a variety of vehicle models, providing a safer, better user experience for drivers.
Industry pioneers use a powerful ADAS image data processing pipeline
As a pioneer in automotive vision safety technology, Mobileye is the first to adopt the new Mali-C78AE ISP in its new generation of EyeQ technology. At the beginning of the development of Mobileye EyeQ Ultra and EyeQ6H, Mobileye chose Mali-C78AE for efficient image data processing, and with the Mali-G78AE GPU, it achieved safe, smooth, real-time and intuitive graphics rendering to meet the demanding automotive needs.
Elchanan Rushinek, Executive Vice President of Engineering at Mobileye, said: "By combining Arm ISP and GPU technology for panoramic visualization capabilities, we can provide a centralized computing solution that enables state-of-the-art ADAS and parking assistance systems with a single SoC. ”
(7870885)