Improved Model Architecture for Multi-Object Detection via Multi -Sensor LiDAR and Camera Data Fusion

Abstract

Research on multi-object detection is becoming increasingly prominent in the field of object recognition because of breakthroughs in deep learning. Camera and LiDAR are sensor technologies utilized for object detection in diverse applications. They provide distinct benefits and drawbacks for object detection, contingent upon the specific scenario and requirements. In this paper, a new methodology is proposed integrating the advantages of both LiDAR and Camera sensors suppressing their disadvantages in detecting pedestrians. The LiDAR 3D points cloud is converted to a depth image before passing through the feature extraction stage. Deep learning Aggregation (DLA-34) with its hourglass shape is used to extract features from the depth image and the 2D image from the camera. Features from both images were fused through a parallel network of Pyramid Split Attention (PSA) mechanism before passing through the Feature Pyramid Network (FPN) for detecting pedestrians. The proposed model was tested on the benchmark NuScenes, KITTI and Waymo datasets. Accuracy, precision, recall, F1-score, and mAP are used to evaluate the proposed model. the proposed model outperformed the previous state of arts in terms of mAP reaching an improvement of 3% for car and truck object detection while reaching 9.5% mAP improvement for motorcycle object detection in Nuscenes dataset. For Waymo Dataset, the proposed model reached an improvement of 2%, in terms of AP for Vehicles detection in Ll difficulty, and 4 % in L2 difficulty. Moreover, in KITTI dataset, the proposed model reached an improvement of 2.6 % in Car detection for Medium difficulty.