Object Detection Enhanced AR Platform for Assembly Instructions

Augmented reality (AR) is beginning to show promise for training workers in industry. However, current methods of making AR instructions for training require technical expertise and high time cost. This thesis presents an AR instruction authoring system for assembly tasks through reducing the AR registration pipeline and keeping the creation process in-situ. We explore the design of an AR interface integrated with an object detection algorithm to accelerate AR registration. We developed an AR-based system to create tutorial content by capturing subject matter expert (SME) environment-object interactions along with voice instructions. To validate the design, a two part evaluation study deployed the system in 3 different real world spatial tasks. The first part showed that expert participants were able to create task instructions faster than other mediums through AR. The second part revealed expert AR instructions can successfully help novice users complete their tasks. The results were used to refine the design into a system that also captures and overlays 2D video in AR.