The description of "smart car = smart phone + four wheels" is simple and crude, but it is not unreasonable.
Today, we drive smart connected cars and use smartphones to do the same, with the ability to connect to the Internet, as well as familiar mobile applications, and the "supercomputer" behind it acts as the "brain" of the car. The difference is that the car has "kinetic energy" and needs to interact with people, cars, and roads, so the perception system plays an important role in intelligent cars.
The automobile perception system is composed of cameras, millimeter wave radar, lidar, etc., of which the camera has the longest history of application in automobile perception, due to low cost, mature technology, and millimeter wave radar and other sensor advantages complement each other, the status will be difficult to be replaced for a long time. At the same time, in order to comply with the trend of automobile intelligence and networking, the threshold of the on-board camera and on-board imaging industry has gradually been raised, including higher image quality, more complex hardware configuration, and higher computing power requirements.
More cameras and applications on the car have become an important symbol of the difference between smart cars and traditional cars, and also an important embodiment of automotive technology innovation.
Let's take a look at some of the car camera applications.
DMS (Driver Monitoring System): From commercial vehicles to passenger cars, growth trends are general
DMS is currently the cabin car camera in recent years the fastest growing application area, with the "two passengers and one danger" and other policies to promote, DMS has gradually become the standard of commercial vehicles, but also will penetrate in passenger cars. Not limited to ride-hailing, DMS capabilities will play an important role in safe driving until L4 and above autonomous driving technology truly arrives.
Based on computer vision technology, dmS cameras can determine whether the driver is distracted and tired by capturing the driver's facial parameters and comparing them with the preset database, thus triggering the early warning system to refocus the driver's attention.
DMS camera mounting positions are generally located on the roof or A-pillar directly above the driver. In addition, some manufacturers also innovatively hide the camera between the steering wheel and the dashboard, which is both beautiful and saves space to a certain extent. For example, the Cadillac super cruise in the following figure:
Previously, the DMS camera in Tesla's cockpit sparked controversy over privacy breaches. Inspired by Mark Zuckerberg's tape on the laptop camera, the automotive electronics industry also began to do the business of camera mask accessories. Such as BYD Han, Celis asked the M5.
The question can obscure the OMS camera
In addition, DMS cameras can not only be used as sensors to monitor the driver's status, but can also be combined with AR HUD products, transparent A-pillars and other applications that need to calculate the driver's line of sight
OMS (Occupant Monitoring System): It has a place in protecting the safety of people and property
OMS (Occupant Monitoring System) is a derivative application of dms function to expand the overall interior space of the car.
In the same way, based on vision technology, OMS realizes the detection of co-drivers, rear occupants, and remnants, bag seat belt detection, passenger posture detection, passenger number detection, especially the detection and reminder of leftovers and children after the owner gets off the bus, which plays a role in personal safety and property safety.
Whether it is DMS or OMS, its current application is mainly still in the early warning stage, with greater expansion and imagination space, can be combined with AD/ ADAS system to achieve personalized body control functions.
For example, combined with algorithms and machine learning, identity authentication and richer human-vehicle interaction are realized through face recognition and emotion recognition. When the driver's identity is recognized, the car seat adjustment, personalized playlist/broadcast recommendation, and recommendation of common navigation destinations will be provided according to the presets or habits of the corresponding individual.
In-car gesture recognition and control: emerging human-vehicle interaction methods, large-scale application is still early
In the past, gesture recognition and manipulation as a "black technology" have caused a wave due to BMW's application.
The 3D camera on the center console is the perceptual hardware basis for gesture recognition and control in the car, and the gesture is analyzed according to the time difference between the transmitted light and the receiver (hand), and the preset gesture in the system is compared to achieve the corresponding operation.
As an emerging way of human-computer interaction, gesture control can replace some touch and click operations. At present, the gesture recognition interaction in the car is still in the early stage of development, mostly used in streaming media control, in the security device control, central control screen, etc. have not been able to replace the physical button, touch.
Byton M-Byte gesture recognition and manipulation
Forward view: ADAS capability with a larger field of view
The camera used for external environment perception is generally distributed in front of the car, behind the car, and on both sides of the body, with an external number of 4-8, the main purpose is to detect the surrounding environment of the body, provide the driver with the surrounding environment information, and also realize the auxiliary driving function to a certain extent, including adaptive cruise, lane line departure warning, pedestrian vehicle collision warning, etc.
The forward camera is generally installed behind the windshield, the number of installations is 1-3, and the configuration can meet the wider field of view, deeper perception ability, and has a high dynamic range.
Tesla front-facing 3 camera combo
Taking Tesla as an example, the front-facing 3 camera combinations are front-view wide field of view (FISHEYE lens with FOV of 120°), main field of view, and narrow field of view camera.
NIO's new model, the ET7, adopts different installation methods, including three cameras in the front, one of which is behind the windshield in the car, and the other two are installed on both sides of the roof outside the car, in order to improve the viewing angle, avoid the next vehicle obscuring pedestrians and obstacles, and improve driving safety.
A camera is installed on each side of the "horn" on the roof of the NIO ET7, and the lidar is in the middle of the roof
Transparent A-pillar: reduces visual field blind spots
In March 2021, the "transparent" A-pillar developed by Nezha Automobile, a brand of United, was officially mass-produced and installed on Nezha U. The "transparent" A-pillar is based on the AR-View system of Nezha Automobile, which uses OLED flexible screen to seamlessly connect the interior of the A-pillar, greatly improving the display contrast while also ensuring the original mechanical strength, with the independently developed intelligent software algorithm and the external high-definition camera to achieve the "transparent" visualization of the A-pillar.
Image source, Autohome
Of course, the device is not just as simple as transmitting the picture back to the A-pillar display through the camera.
The "transparent" A-pillar equipped by Nezha U is based on self-developed core algorithms such as "3D reconstruction and perspective transformation of eyebrow position" and "dynamic mapping of blind spots of depth information", which can track the driver's eyes through the cabin camera and calculate the eyebrow position, so as to confirm the display perspective angle of the A-pillar display and display it in real time, so that the driver can always maintain a clear field of vision.
360° Panoramic Surround View System + Chassis Perspective Imaging System: Escort for safe autonomous parking
The 360° panoramic surround view system is generally composed of 4-6 high dynamic range, high-resolution cameras, distributed in front of the car, behind the car, and on both sides of the body. The system forms a bird's-eye view of the image information collected by the camera around the body and projects it onto the car's central control, and the field of view covers the car body in all directions.
Land Rover reversing image
Under the trend of automotive intelligence, the 360° panoramic surround view system has also become an important technical basis for autonomous parking and one-click activation of vehicle functions, which usually need to be combined with millimeter-wave radar to achieve more accurate road and obstacle detection.
In addition to the 360° panoramic surround view system, some automakers have also proposed an upgraded "chassis perspective" imaging system to achieve higher safety parking. The principle is to project the image information in front of the vehicle and under the front of the vehicle in real time on the central touch screen, clearly presenting a vast 180° virtual view. At present, most of the chassis perspective imaging systems are used in SUV models, and other models are gradually following suit.
The combination of the 360° panoramic surround view system + the 180° chassis perspective image system further eliminates the blind spot of immediate driving and avoids risks for the realization of autonomous parking function. For example, the 540° panoramic intelligent image equipped with the Lynk & Co 06 released in early 2021 is based on the 360° panoramic surround view system and the 180° chassis perspective image system.
The Lynk & Co 06 is equipped with a 540° panoramic intelligent imaging system that enables autonomous parking and starting of vehicle functions
CMS (Electronic Rearview Mirror): The advantages and concerns are clear, with a focus on regulations
Electronic rearview mirror is the relatively avant-garde application of vehicle cameras, is the product of the electronic of traditional equipment (physical rearview mirrors) of automobiles, Toyota's Lexus, Audi have released a front-mounted electronic rearview mirror model.
Audi e-tron electronic rearview mirrors
For example, Lexus' electronic rearview mirror completely replaces the physical rearview mirror through the camera, and uses the cameras on both sides of the car to collect road information on both sides. At the same time, a liquid crystal display screen is added on each side of the front row of the cockpit to display image information.
Compared with traditional rearview mirrors, electronic rearview mirrors have the advantages of small and beautiful appearance, reduced wind resistance, wider line of sight, and in some cases (such as rain and snow) the display display is not affected.
The disadvantages of the electronic rearview mirror are also obvious, including the increase in cost, the LCD screen is easy to lead to the driver's visual fatigue, picture delay and so on. For automakers, such an emerging device will pose greater challenges for automotive terminal processing systems.
Policy is currently another important factor restricting the development of electronic rearview mirrors, for now, the European Union, Japan has allowed vehicles without physical rearview mirrors on the road, the United States is also promoting relevant regulations. Domestic policies still need to wait for the GB15084 expansion to be completed, and industry insiders said that this standard will be passed in the middle of next year.
Of course, compared to CMS, streaming mirrors that incorporate rearview cameras are relatively easier to implement.
Cameras allow consumer electronics entertainment to "get in the car" and cater to millennials
With the entry of more and more Internet companies and new forces, the application of on-board cameras carries more Internet "genes", transplants the functions of consumer electronics to the car and attaches social play, such as taking photos in the car, video calls, and even shooting outside the car.
As a typical example, the Xiaopeng G3 is equipped with a 360° panoramic camera on the roof, which is not only used for the detection of the surrounding environment of the car, but also brings the Internet gameplay to the car - the camera can be controlled through the mobile phone and the car central control to control the camera lifting and 360 ° rotation, and supports one-click shooting of scenery along the way and sharing the community.
Similarly, the zhiji car that debuted earlier this year also carried forward this idea, and the roof is equipped with a carlog social lens. Carlog, as the name suggests, the concept is similar to vlog, plog, blog, all have the meaning of "recording life", the difference is in the realization of the scene.
In addition, the Internet-style trendy concept of car + drone has recently re-entered the public eye. Not long ago, Xiaopeng announced that the new model P5 supports users to choose DJI drones when buying cars, which can realize the linkage between drones and cars. This undoubtedly caters to the younger generation's deep yearning for tide and coolness once again.
At present, there are still some time before the official delivery of Xiaopeng P5, according to the imagination of the industry in the past, the general logic of the car and UAV linkage on the consumer side application is to put the control interface on the central control screen, detect the road conditions in front of the road or in the field or shoot the movement status of the car and the owner to share to the community and so on.
Reproduced from the Intelligent Car Club, the views in the article are only for sharing and exchange, and do not represent the position of this public account, such as copyright and other issues, please inform, we will deal with it in a timely manner.
-- END --