Abstract
Objective Light detection and ranging (LiDAR) plays an important role in autonomous driving. Even if it is expensive, the number of equipped laser beams is still small, which results in a sparse point cloud density. The sparsity of the LiDAR point cloud makes 3D object detection difficult. The camera is another vital sensor used in autonomous driving because of the mature image recognition methods and its competitive price compared with LiDAR. However, it does not perform as well as LiDAR in 3D object detection task. To perceive the surrounding environment better, this study proposes a LiDAR data enhancement algorithm based on pseudo-LiDAR point cloud correction method to increase the density of LiDAR point cloud, thereby improving the accuracy of 3D object detection. This method has a wide application prospect because it is a general method to densify the point cloud and improve the detection accuracy, which does not depend on specific 3D object detection network structures. Method The algorithm can be divided into four steps. In the first step, the depth map is generated on the basis of the stereo RGB images by using depth estimation methods, such as pyramid stereo matching network (PSMNet) and DeepPruner. The approximate 3D coordinates of each pixel corresponding to the LiDAR coordinate system are calculated according to the camera parameters and depth information. The point cloud formed by approximate 3D coordinates is usually called pseudo-LiDAR point cloud. In the second step, the ground points in the original point cloud are removed by ground segmentation method because they will disturb the surface reconstruction process in the following step. In order to improve the performance of ground segmentation, this study designs an iterative random sample consensus (RANSAC) algorithm. A register is used in the iterative RANSAC algorithm to store the points extracted in each iteration, which are planar point cloud but not ground point cloud. The register ensures that the next iteration of RANSAC will not be affected by these non-ground planar points. The iterative RANSAC algorithm performs better than the normal RANSAC algorithm in complex scenarios where the number of points in the non-ground plane is larger than that in a ground plane or there exist multiple ground planes with different angles of inclination. In the third step, the original point cloud is inserted into a k-dimensional tree (KDTree) after ground segmentation. Thereafter, several neighboring points of each point in the pseudo-LiDAR point cloud are obtained by searching in the KDTree. On the basis of these neighboring points, surface reconstruction is performed to catch the local feature around the pseudo-LiDAR point. The Delaunay triangulation surface reconstruction method, which is considered as the optimal surface reconstruction algorithm by many researchers, is used in this step. The result of surface reconstruction is represented as several tiny triangles that cover the whole surface in 3D space. If the distance between the current and last processed approximate 3D point is within the KDTree searching radius, the surface reconstruction result of the last processed point is directly regarded as the surface reconstruction result of the current point to save time by skipping the KDTree searching and surface reconstruction process. Furthermore, because the process of KDTree searching and surface reconstruction is independent for each pseudo-LiDAR point, parallel computation based on OpenMP is used to speed up this step. In the fourth step, the precise 3D coordinates of pseudo-LiDAR point cloud corresponding to the LiDAR coordinate system are derived by the designed computational geometry method. In the computational geometry method, two different depth values are set for each pixel to obtain two points in the 3D coordinate system. The two points determine the line of the light path of this pixel. Then, the precise 3D coordinate of this pixel is considered as the closest point from the origin among all of the intersection points of the light path line and reconstruction triangle surfaces. This computational geometry method realizes the function of pseudo-LiDAR point coordinate correction. To prevent the loss of accuracy, the method avoids using division in the calculation process based on inequality analysis. Finally, the precise 3D points generated in step four are merged with the origin point cloud scanned by LiDAR to obtain the dense point cloud. Result After the densification, the objects in the point cloud have more complete shapes and contours than before, which means that the characteristics of objects are more obvious. In order to further verify the validity of this data enhancement method, the aggregate view object detection (AVOD) and aggregate view object detection with feature pyramid network (AVOD-FPN) methods are implemented to check whether the average precision of 3D object detection on the dense point cloud is higher than that of the original point cloud. The data enhancement algorithm was used on KITTI (Karlsruhe Institute of Technology and Toyota Technological Institute) dataset to obtain a dataset with dense LiDAR point clouds. Then, 3D object detection methods were implemented on the original LiDAR point cloud and dense LiDAR point cloud. After using this data enhancement method, the AP3D-Easy of AVOD increased by 8.25%, and the APBEV-Hard of AVOD-FPN increased by 7.14%. Conclusion A vision-based LiDAR data enhancement algorithm was proposed to increase the density of LiDAR point cloud, thereby improving the accuracy of 3D object detection. The experimental results show that the dense point cloud has good visual quality, and the data enhancement method improves the accuracy of 3D object detection on KITTI dataset.
