An Exploration of the Virtual Digital Twin Capture for Spatial Tasks and its Applications

Our generation is currently at the juncture of the fourth industrial revolution - Industry 4.0. Emergent technology such as Augmented Reality (AR), Internet of Things (IoT), Artificial Intelligence (AI), cloud computing, big data, and ore are at the center of this. Amidst all these, the concept of  digital Twinning is a promising technology for realizing Industry 4.0. Simply put, a Digital Twin is a virtual representation of a real task, action, or object. This thesis explores the parameters and details required to generate a Digital Twin. Using these insights, we propose two applications that utilize digital twinning - EditAR and AnnotateXR. EditAR is an AR workflow for authoring kinesthetic instructions for spatial tasks. AnnotateXR is an Extended Reality (XR) workflow for automating data annotation to support multiple Computer Vision (CV) applications. We evaluate these systems through user studies and report the results on the usability and viability of these workflows. From an evaluation study, EditAR received an average system usability score (SUS) of 82.0. Over the course of a user study, using AnnotateXR, users were able to generate a total of 112,737 semantically segmented images and 144 videos annotated for action segmentation in 66.55 minutes. AnnotateXR received an average SUS score of 91.0.