Recently, on the authoritative evaluation set of autonomous driving nuScenes, the radar and image pre-fusion algorithm PAI3D proposed by the Jingdong Logistics autonomous driving team has obtained the world's first result in multi-sensor fusion 3D object detection (using any sensor, no additional data), which means that the autonomous driving technology of JD Logistics Terminal Distribution has been at the world's leading level.
PAI3D is a 3D target detection algorithm proposed by JD Logistics Autonomous Driving based on rich experience in terminal distribution scenarios, which fully considers the identification accuracy, calculation efficiency, sensor calibration error and other issues. In the actual use of the car end, it effectively solves the problems of identification instability caused by sparse distant point clouds, missed inspection caused by special material absorption point clouds, difficult identification of small obstacles, and inaccurate depth estimation of 3D target detection only relying on monocular vision, which strongly supports the large-scale operation of Jingdong Logistics intelligent delivery vehicles.
The nuScenes dataset is a very large autonomous driving dataset published in March 2019 by Motional (former nuTonomy), a joint venture between Hyundai Motor Group and Aptiv, which has 1,000 driving scenes, 1.4 million images, 390,000 lidar point clouds, 1.4 million millimeter wave radar frames and 1.4 million obstacle truth frames marked from 40,000 keyframes. It is the first large-scale autonomous driving dataset, which is derived from the entire sensor suite of autonomous vehicles (6 cameras, 1 lidar, 5 radars, GPS, IMU), which is 7 times the size and difficulty of the KITTI dataset in terms of labeling data volume, and its scale and difficulty exceed that of kitti, Udacity and other public datasets. At the same time, huawei, sensetime, Samsung, University of Science and Technology of China, Shanghai Jiao Tong University, Beihang University, Beijing Institute of Technology, University of Chinese, University of Chinese, University of Texas Austin, Nanyang Technological University, Johns Hopkins University and other well-known enterprises and research institutions at home and abroad are also involved in nuScenes multi-sensor fusion 3D object detection.
At present, in the automatic driving L4 scheme, perception mainly relies on lidar, but lidar has some defects, such as sparse information at a distance and no color information. Image information can compensate for these deficiencies, but also lack depth information. Therefore, in the technology of automatic driving, how to use multimodal sensor information to design 3D detection algorithms is the core basis of the automatic driving perception system. The PAI3D algorithm proposed by JD Logistics has achieved good recognition accuracy by fusing image and point cloud information at the level of advanced features and low-level features, taking advantage of its strengths and avoiding its weaknesses. At the same time, PAI3D takes into account the calculation force distribution of the car end, the sensor calibration error and other factors, and has good practicality. In addition, PAI3D also effectively solves a series of problems encountered in complex open road operations, such as the identification instability caused by the sparse point cloud in the distance, the missed inspection caused by the absorption point cloud of special materials, the difficulty of identifying small obstacles, and the inaccurate estimation of the depth of the detection of only monocular vision 3D objects, which improves the accuracy of the location and category estimation of obstacles, and reduces the false detection and leakage detection of obstacles.
PAI3D has achieved the world's first result in multi-sensor fusion 3D object detection in the nuScenes dataset, which can bring the following three advantages to the autonomous driving technology of JD Logistics' terminal distribution. The first is to realize module multiplexing and task reuse through multi-sensor fusion. With the increase of operational scenarios, the perception system of autonomous driving needs to recognize more and more elements, PAI3D can integrate information across sensor modalities, and make full use of image and point cloud semantic segmentation information to achieve 1+1 > 2 effects. Second, it fully considers the problem of computing resource allocation, and has the practicality of flexible deployment to the vehicle-side heterogeneous parallel computing platform. In the future in-vehicle systems, it is necessary to consider problems such as time-sharing multiplexing and heterogeneous parallel computing. PAI3D's converged approach can be flexibly deployed as computing resources change. Third, calibration errors and motion compensation are insensitive, in daily operation, the relative position of sensors changes over time, and the physical principles of lidar and cameras lead to a degree of space-time inconsistency between the two. PAI3D fully considers this problem, and can tolerate calibration errors and motion compensation errors within certain limits, and has strong fault tolerance. It is based on these three advantages that not only highlights the technical strength of JD Logistics in the field of automatic driving, but also means that JD Logistics' terminal distribution autonomous driving technology has been at the world's leading level.
In recent years, driven by long-term technology investment and innovation, the core competitiveness of Jingdong Logistics's software, hardware and system integration of trinity supply chain logistics technology has been continuously upgraded, and technical products and solutions covering key links of the supply chain such as parks, warehousing, sorting, transportation and distribution have been formed to improve forecasting, decision-making and intelligent execution capabilities, help more customers fully optimize the supply chain network, improve digital intelligence and operational efficiency.
Text/Beijing Youth Daily reporter Wen Jing
Edit/Field