Mobile Mapping Systems Camera–LiDAR Data Registration for Mitigating GNSS/INS Trajectory Perturbations

<p dir="ltr">Wheeled Mobile Mapping Systems (MMS), equipped with LiDAR scanners, cameras, and integrated Global Navigation Satellite System/Inertial Navigation System (GNSS/INS) units, are widely used in urban planning, high-definition map generation, and infrastructure monitoring. Accurate registration between wheeled MMS camera and LiDAR data–which is ensured by precise system calibration and GNSS/INS trajectory–is essential for effective data fusion, enabling advanced analyses to address the needs of these applications. Although wheeled MMS undergo high-precision initial calibration, environmental factors affecting sensor calibration and/or GNSS/INS trajectory accuracy can cause misalignment between imagery and LiDAR data. System’s calibration parameters, including mounting parameters and sensors’ Interior Orientation Parameters (IOP), may be affected by sensor aging and environmental conditions, which can degrade the initial calibration. Additionally, data collection along transportation corridors presents challenges due to interference from neighboring traffic, bridges, buildings, and canopy, which can cause GNSS signal occlusions and accuracy reduction of the GNSS/INS trajectory. The GNSS/INS trajectory errors are more common than system calibration errors. This research identifies trajectory issues by analyzing the misalignment between image and LiDAR data. The research introduces a novel registration approach to improve the alignment of image and LiDAR data by establishing an appropriate transformation function, automatically extracting lane markings as common primitives, and developing a similarity measure suited to the characteristics of these primitives. These components are integrated into an automated primitive matching and learning-based optimization strategy that solves for the transformation function parameters. The proposed learning-based registration algorithm operates effectively in both urban and highway environments, offering a robust solution for accurate camera-LiDAR alignment across diverse settings. Furthermore, an analysis of the registration results and estimated poses of stereo cameras mounted on top of the wheeled MMS across sequential image sets–before and after the registration process–is introduced to help identify the root causes of misalignments, whether they arise from GNSS/INS trajectory errors or calibration inaccuracy. This registration algorithm has been evaluated on several datasets using quantitative metrics, such as the mean of the minimum Euclidean distances and Intersection over Union (IoU). Following this registration, in most cases, the misalignment between projected LiDAR and image features is reduced from hundreds or tens of pixels to less than couple of pixels, with IoU improvements exceeding 50%.</p>