AUTOMATED BRIDGE INSPECTION IMAGE LOCALIZATION AND RETRIEVAL BASED ON GPS-REFINED SIMILARITY LEARNING

The inspection of highway bridge structures in the United States is a task critical to the national transportation system. Inspection images contain abundant visual information that can be exploited to streamline bridge assessment and management tasks. However, historical inspection images often go unused in subsequent assessments as they are disorganized and unlabeled. Further, due to the lack of GPS metadata and visual ambiguity, it is often difficult for other inspectors to identify the location on the bridge where past images were taken. While many approaches are being considered toward fully- or semi-automated methods for bridge inspection, there are research opportunities to develop practical tools for inspectors to make use of those images already in a database. In this study, a deep learning-based image similarity technique is combined with image geolocation data to localize and retrieve historical inspection images based on a current query image. A Siamese convolutional neural network (SCNN) is trained and validated on a gathered dataset of over 1,000 real world bridge deck images collected by the Indiana Department of Transportation. A composite similarity (CS) metric is created for effective image ranking and the overall method is validated on a subset of eight bridge’s images. The results show promise for implementation into existing databases and for other similar structural inspections, showing up to an 11-fold improvement in successful image retrieval when compared to random image selection.