OBJECT EXPLORATION, CHARACTERIZATION, AND RECOGNITION BASED ON TACTILE SENSING

Tactile sensing is an essential human ability for understanding their surroundings. It allows humans to detect and manipulate objects that are concealed or difficult to see in low-light settings. Further, tactile sensing enables people to comprehend object and surface properties that cannot be obtained through visual feedback alone. This is achieved with gentle touches, enabling tactile exploration of fragile, sensitive objects, or living organisms. This capability could be transferred to robots through suitable hardware and algorithms. Nevertheless, current tactile sensors and skills for robotics are not comparable to the tactile sense of humans, thus resulting in inferior characterization of scenes and a risk of altering object states.

To address these limitations, this dissertation proposes a novel framework for robot active tactile exploration and object characterization. The framework combines bioinspired soft sensors and minimally invasive tactile exploration strategies to minimize perturbations to objects. This framework was achieved by: (1) an ultrasensitive whisker sensor that enables object characterization with minimal interaction forces; (2) autonomous tactile exploration skills to localize objects and then characterize their shape and surface properties; and (3) machine learning techniques to analyze contact information gathered by our tactile sensors, enabling the understanding of object attributes by tactile sensing alone. 

Experiments were conducted to validate the effectiveness of the framework. In terms of object localization efficiency, informative path planners and contour exploration patterns outperformed baseline methods. Furthermore, the whisker sensor was successfully employed to characterize object surface and liquid properties. Finally, the features found through the characterization process allowed for successful classification by machine learning techniques. These results indicate that the proposed framework can effectively gather multimodal features from environments while maintaining the safety of objects.