Purdue University Graduate School
JaredPurdueThesisNew3.pdf (23.4 MB)

The Amazing Race: Robot Edition

Download (23.4 MB)
posted on 2021-04-29, 15:04 authored by Jared JohansenJared Johansen
We describe a new task called The Amazing Race: Robot Edition. In this task, the robot is placed in a real, unknown environment, without a map, and asked to find a designated location. It will need to explore its surroundings, find and approach people, engage them in a dialogue to obtain directions to the goal, and follow those directions to the hallway with the goal. We describe and implement a variety of robotic behaviors that performs each of these functions. We test these in the real world in test environments that were distinct from the training environments where we developed our methods and trained our models. Additionally, these test environments were completely unmodified and reflect the state of the real world.
First, we describe how our robotic system solves this problem where the environment is constrained to a single floor or a single building. We demonstrate that we are able to find a goal location in never-before-seen environments. Next, we describe a machine-learned approach to the dialogue and components of our system to make it more robust to the diversity and noisiness of navigational instructions someone may provide.


NRI: Collaborative Research: RobotSLANG: Simultaneous Localization, Mapping, and Language Acquisition

Directorate for Computer & Information Science & Engineering

Find out more...

NCS-FO: Neuroimaging to Advance Computer Vision, NLP, and AI

Directorate for Computer & Information Science & Engineering

Find out more...

IARPA: DIVA: Deep Intermodal Video Analytics

Siemens Corporation, Corporate Technology


Degree Type

  • Doctor of Philosophy


  • Electrical and Computer Engineering

Campus location

  • West Lafayette

Advisor/Supervisor/Committee Chair

Jeffrey Mark Siskind

Additional Committee Member 2

Robert Givan

Additional Committee Member 3

Ronnie B Wilbur

Additional Committee Member 4

Thomas Talavage

Usage metrics



    Ref. manager