Purdue University Graduate School

File(s) under embargo

Reason: pending patent





until file(s) become available

Object Detection Enhanced AR Platform for Assembly Instructions

posted on 2022-09-09, 14:58 authored by Hank HuangHank Huang
Augmented reality (AR) is beginning to show promise for training workers in industry. However, current methods of making AR instructions for training require technical expertise and high time cost. This thesis presents an AR instruction authoring system for assembly tasks through reducing the AR registration pipeline and keeping the creation process in-situ. We explore the design of an AR interface integrated with an object detection algorithm to accelerate AR registration. We developed an AR-based system to create tutorial content by capturing subject matter expert (SME) environment-object interactions along with voice instructions. To validate the design, a two part evaluation study deployed the system in 3 different real world spatial tasks. The first part showed that expert participants were able to create task instructions faster than other mediums through AR. The second part revealed expert AR instructions can successfully help novice users complete their tasks. The results were used to refine the design into a system that also captures and overlays 2D video in AR.




Degree Type

  • Master of Science in Electrical and Computer Engineering


  • Electrical and Computer Engineering

Campus location

  • West Lafayette

Advisor/Supervisor/Committee Chair

Karthik Ramani

Additional Committee Member 2

Alexander J. Quinn

Additional Committee Member 3

Benes Bedrich