Details the application of artificial intelligence in autonomous driving

Heavy dry goods, the first time to deliver

Source: Smart Car Technology, Editor: Datawhale

With the rapid development of technology, cloud computing, big data, artificial intelligence and some new terms have entered the public's field of vision, artificial intelligence is another technological revolution after human beings enter the information age is receiving more and more attention. As an extension and application of artificial intelligence technology in the automotive industry and transportation field, unmanned driving has been closely watched by industry, academia and even the national level in recent years.

Self-driving cars rely on artificial intelligence, visual computing, radar, monitoring devices, and GPS to work together to allow computers to operate motor vehicles autonomously and safely without any human initiative. Autonomous driving technology will become a new development direction for future cars.

This article will mainly introduce the application field of artificial intelligence technology in autonomous driving, and make a simple analysis of the development prospects of automatic technology.

Artificial intelligence is a science that started late but developed rapidly. Since the 20th century, scientific workers have been seeking ways to give intelligence to robots. The concept of modern artificial intelligence was developed from the British scientist Turing's search for intelligent machines, until 1937 Turing's paper "Ideal Automata" gave artificial intelligence a strict mathematical definition, and many of the problems actually to be handled in the real world cannot be simply numerical calculations, such as speech understanding and expression, graphic image and sound understanding, medical diagnosis and so on.

In 1955, Newell and Simon's Logic Theorist proved 38 of the first 52 theorems in Mathematical Principles. Simon asserts that they have solved the question of how systems of matter acquire the nature of the mind (a thesis that later became known in the field of philosophy as "strong artificial intelligence"), arguing that machines have the ability to think logically like humans. In 1956, "artificial intelligence" (AI) was proposed by John McCarthy of the United States, and after an early stage of exploration, artificial intelligence developed in a more systematic direction, becoming an independent discipline.

In the 1950s, the research of artificial intelligence began with the game game as the object; in the 1960s, the research on the search method to solve general problems was the mainstay; in the 1970s, artificial intelligence scholars conducted effective artificial intelligence research; in the 1980s, the research on uncertain reasoning, non-monotonous reasoning, and theorem reasoning methods began; in the 1990s, breakthroughs were made in basic research such as knowledge representation, machine learning, and distributed artificial intelligence.

An overview of the application of artificial intelligence in autonomous driving technology

Artificial intelligence has developed for sixty years, several ups and downs, and now ushered in another boom, deep learning, computer vision and natural language understanding and other aspects of the breakthrough, making many once fanciful applications possible, driverless cars are one of them. As an extension and application of artificial intelligence and other technologies in the automotive industry and transportation fields, unmanned driving has been closely watched by industry, academia and even the national level in recent years. At present, artificial intelligence has also been widely used in automotive autonomous driving technology.

Self-driving cars rely on artificial intelligence, visual computing, radar, monitoring devices and global positioning systems to cooperate, it is a set of environmental perception, planning decision-making, multi-level assisted driving and other functions in one of the integrated system, it concentrates on the use of computers, modern sensing, information fusion, communications, artificial intelligence and automatic control and other technologies, is a typical high-tech complex.

This kind of car can "think", "judge" and "walk" like people, so that computers can automatically and safely operate motor vehicles without any human active operation. According to the classification of SAE (American Society of Automotive Engineers), it is divided into five levels: driver assistance, partial automatic driving, conditional automatic driving, highly automated driving, and fully automated driving.

Phase I: Driver Assistance aims to assist the driver, including providing important or beneficial driving-related information and issuing clear and concise warnings when a situation begins to become critical. At this stage, most ADAS active safety assistance systems enable vehicles to perceive and intervene in operation. For example, anti-lock braking system (ABS), electronic stability control (ESC), lane departure warning system, frontal collision warning system, blind spot information system, etc., at this time, the vehicle can know the surrounding traffic conditions through cameras and radar sensors, and then make warnings and interventions.

The second stage: some autonomous vehicles through the camera, radar sensors, laser sensors and other equipment to obtain road and surrounding traffic information, the vehicle will provide driving support for the steering wheel and acceleration and deceleration of a number of operations, in the driver received a warning but failed to take timely corresponding action can automatically intervene, other operations handed over to the driver, to achieve man-machine co-driving, but the vehicle does not allow the driver's hands off the steering wheel. Examples include Adaptive Cruise Control (ACC), Lane Keeping Assist (LKA), Automatic Emergency Braking (AEB) System, Lane Departure Warning (LDW), etc.

The third stage: conditional automatic driving by the automatic driving system to complete the driving operation, according to the road conditions, if necessary to issue a system request, must be handed over to the driver to drive.

Stage 4: Highly automated driving is done by the autonomous driving system, and according to the system's request, the driver can not take over the vehicle. The vehicle can already complete the automatic driving, once the automatic driving system can not be overwhelmed, the vehicle can also adjust to complete the automatic driving, the driver does not need to interfere.

Phase V: The ideal form of fully autonomous driving, passengers only need to provide the destination, regardless of any road conditions, any weather, the vehicle can achieve automatic driving. This level of automation allows passengers to engage in activities such as computer work, rest and sleep, and other recreational activities, without the need to monitor the vehicle at any time.

Realization of autonomous driving

To achieve automatic driving, the vehicle must go through three major links:

First, perception. That is to say, to let the vehicle acquire, different systems need to be different types of vehicle sensors, including millimeter wave radar, ultrasonic radar, infrared radar, laser radar, CCD \CMOS image sensor and wheel speed sensor to collect the working state of the vehicle and its parameter changes.

Second, processing. That is, the brain analyzes and processes the information collected by the sensor, and then outputs the control signal to the control device.

Third, execution. According to the signal output of the ECU, let the car complete the action execution. Each of these links is inseparable from the foundation of artificial intelligence technology.

The application of artificial intelligence in autonomous driving positioning technology

Positioning technology is the basis for autonomous vehicle driving. At present, commonly used technologies include line navigation, magnetic navigation, wireless navigation, visual navigation, navigation, laser navigation, etc.

Among them, magnetic navigation is the most mature and reliable solution at present, and most existing applications use this navigation technology. Magnetic navigation technology by burying magnetic signs in the lane to provide vehicles with lane boundary information, magnetic materials have good environmental adaptability, it can be adapted to rainy days, ice and snow cover, insufficient light or even no light, the disadvantage is that the current road facilities need to make large changes, the cost is higher. At the same time, magnetic navigation technology cannot predict the obstacles in front of the lane, so it is impossible to use it alone.

Visual navigation has lower infrastructure requirements and is considered the most promising method of navigation. Visual methods in highway and urban environments have received a great deal of attention.

Application of artificial intelligence in image recognition and perception of autonomous driving

Driverless cars rely on sensors for perception. At present, the performance of sensors is getting higher and higher, the size is getting smaller and smaller, and the power consumption is getting lower and lower, and its rapid development is an important driver of the unmanned driving boom. In turn, unmanned driving has put forward higher requirements for on-board sensors and promoted their development.

Sensors for autonomous driving can be divided into four categories:

Radar sensors

Mainly used to detect the direction, distance and movement speed of obstacles within a certain range (such as vehicles, pedestrians, road shoulders, etc.), commonly used vehicle radar types are lidar, millimeter wave radar and ultrasonic radar. Lidar has high accuracy, wide detection range, but high cost, such as the cost of 64-line lidar on the google unmanned roof of more than 700,000 yuan; millimeter-wave radar cost is relatively low, the detection distance is far, widely used by car companies, but with lidar than the accuracy is slightly lower, the viewing angle is smaller; ultrasonic radar cost is the lowest, but the detection distance is close, the accuracy is low, can be used for low-speed collision warning.

Vision sensors

It is mainly used to identify lane lines, stop lines, traffic lights, traffic signs, pedestrians, vehicles, etc. Commonly used are monocular cameras, binocular cameras, infrared cameras. The cost of vision sensors is low, and there are many related research and products, but visual algorithms are susceptible to lighting, shadows, defacement, and occlusion, and accuracy and robustness need to be improved. Therefore, image recognition, as one of the fields widely used in artificial intelligence technology, is also a research hotspot in the field of driverless cars.

Positioning and position sensors

It is mainly used for real-time high-precision positioning and posture perception, such as obtaining latitude and longitude coordinates, speed, acceleration, heading angle, etc., generally including global satellite positioning system (GNSS), inertial equipment, tachometer, odometer, etc. At present, the commonly used high-precision positioning method in China uses differential positioning equipment, such as RTK-GPS, but it is necessary to set up a fixed differential base station, the application distance is limited, and it is susceptible to buildings and trees. In recent years, the surveying and mapping departments of many provinces and cities have set up a continuous operation reference station system (CORS) equivalent to a fixed differential base station, such as Liaoning, Hubei, Shanghai, etc., to achieve a wide range of positioning signal coverage, and this infrastructure construction provides a strong technical support for intelligent driving. Positioning technology is the core technology of unmanned driving, because with location information, you can use rich prior knowledge of geography, maps, etc., and you can use location-based services.

Body sensors

From the vehicle itself, information such as vehicle speed, wheel speed, gear and other vehicles themselves is obtained through the vehicle network interface.

The application of artificial intelligence in deep learning for autonomous driving

Driver cognition depends on the brain, and the "brain" of driverless cars is the computer. The computer in the unmanned vehicle is slightly different from the desktop and notebook we commonly use, because the vehicle will encounter bumps, vibrations, dust and even high temperatures when driving, and the general computer cannot run in these environments for a long time. Therefore, unmanned vehicles generally use computers in the industrial environment - industrial computers.

The industrial computer runs the operating system, and the operating system runs unmanned software. Figure 1 shows the software system architecture of a driverless car. Above the operating system are the support modules (in this case, the modules refer to computer programs), which provide basic services to the upper software modules.

Support modules include: a virtual switching module for inter-module communication; a log management module for logging, retrieval and playback; a process monitoring module, which is responsible for monitoring the running status of the entire system, prompting the operator and automatically taking corresponding measures if a module is not running properly; and an interactive debugging module, which is responsible for developers interacting with unmanned systems.

Details the application of artificial intelligence in autonomous driving

Figure: Software system architecture of a driverless car

In addition to knowing the outside world, machines must also be able to learn. Deep learning is the basis for the success of driverless technology, and deep learning is an efficient machine learning method derived from artificial neural networks. Deep learning can improve the time efficiency of cars to identify roads, pedestrians, obstacles, etc., and ensure the correct rate of recognition. After training with a large amount of data, the car can convert the collected graphics, electromagnetic waves, and other information into usable data, using deep learning algorithms to achieve autonomous driving.

When driverless cars collect data through radar, etc., the original training data should first be preprocessed. Calculate the mean and standardize the mean of the data, do principal component analysis on the original data, and use PCA whitening or ZCA whitening. For example: the time data collected by the laser sensor is converted into the distance between the car and the object; the photo information taken by the on-board camera is converted into the judgment of the roadblock, the judgment of the traffic light, the judgment of the pedestrian, etc.; the data detected by the radar is converted into the distance between the various objects.

Applying deep learning to driverless cars consists of the following steps:

1. Prepare the data, preprocess the data and then select the appropriate data structure to store the training data and test the tuple;

2. Enter a large amount of data to learn unsupervised learning of the first layer;

3. Cluster the data through the first layer, divide the similar data into the same category, and make random judgments;

4. Use supervised learning to adjust the threshold of each node in the second layer to improve the correctness of the data input of the second layer;

5. Use a large amount of data to perform unsupervised learning on each layer of the network, and train only one layer at a time with unsupervised learning, and use its training results as input to its higher layer.

6. After typing, use supervised learning to adjust all layers.

The application of artificial intelligence in the sharing of information on autonomous driving

First, wireless networks are used to share information between cars. Through a dedicated channel, a car can share its location and road conditions in real time with other cars in the team, so that the automatic driving system of other vehicles can make corresponding adjustments after receiving the information.

Secondly, it is 3D road condition sensing, the vehicle will combine ultrasonic sensors, cameras, radar and laser ranging and other technologies to detect the terrain within about 5 meters in front of the car, determine whether the front is asphalt road or gravel, grass, beach and other road surfaces, and automatically change the car settings according to the terrain.

In addition, the car will be able to automatically change speed, once the terrain is detected to change, it can automatically slow down, the road surface returns to normal, and then return to the original state.

The amount of traffic information collected by car information sharing will be very large, and if this data is not effectively processed and utilized, it will be quickly obliterated by information. Therefore, it is necessary to use data mining, artificial intelligence and other methods to extract effective information, and filter out useless information at the same time. Considering that the information that needs to be relied on during the driving process of the vehicle has a great correlation of time and space, some information needs to be processed in a very timely manner.

Advantages of artificial intelligence applied to autonomous driving technology

AI algorithms focus more on learning functions, while other algorithms focus more on computational functions. Learning is an important embodiment of intelligence, learning function is an important feature of artificial intelligence, at this stage most artificial intelligence technology is still in the stage of learning. As mentioned earlier, unmanned driving is actually human-like driving, which is a smart car that learns from human drivers how to perceive the traffic environment, how to use existing knowledge and driving experience to make decisions and plans, and how to skillfully control the steering wheel, throttle and brakes.

From the perspective of perception, cognition and behavior, the perception part is the most difficult, and the artificial intelligence technology is the most applied. Perception technology, which relies on sensors, such as cameras, is favored in industry due to its low cost. An Israeli company called Mobileye has done a very good job in the field of traffic image recognition, which can complete traffic marking identification, traffic light recognition, pedestrian detection, and even distinguish between bicycles, cars or trucks in front of it through a camera.

The successful application of artificial intelligence technology in the field of image recognition is deep learning, and in recent years, researchers have trained image samples through convolutional neural networks and other deep learning models, which has greatly improved the recognition accuracy. Mobileye's current achievements are due to the company's early research on deep learning as a core technology. In terms of cognition and control, it mainly uses traditional machine learning techniques in the field of artificial intelligence to build driver models by learning the driving behavior of human drivers and learning how people drive cars.

Challenges and prospects for driverless technology

In the context of the current increasingly bad traffic situation, the commercial prospects of "driverless" cars are still constrained by many factors.

The main ones are:

1. Regulatory Obstacles

2. Common agreements are established between different brand models, and the industry lacks norms and standards

3. Basic road conditions, identification and information accuracy, security of information networks

4. Unaffordable and high costs

In addition, one of the biggest features of "driverless" cars is that the vehicle is networked and the degree of informatization is extremely high, which also poses a great challenge to the safety of computer systems. Once the computer program is confused or the information network is invaded, how to continue to ensure the safety of their own vehicles and other vehicles around them is also a problem that needs to be solved urgently in the future. Although there are still many challenges in unmanned driving technology, but unmanned driving is difficult to perceive, the emphasis is on "learning", the level of unmanned driving technology will surpass humans sooner or later, because stability, accuracy and speed are the innate advantages of machines, and humans cannot compare with them.

Driving is sometimes not a burden, but rather a pleasure, reflecting the ability of humans to push their limits. The author believes that complete unmanned driving may be a bit far away, but with the improvement of machine learning algorithms and the mining of applications, more grounded human-machine harmony is just around the corner. No matter how many difficulties there are on the road of autonomous driving, I believe that there will always be a day when it appears on the road of the city, and the development of technology is full of passion and motivation. In the near future, perhaps autonomous driving will become mainstream.

Details the application of artificial intelligence in autonomous driving

Read on