The Amazing Race: Robot Edition

We describe a new task called The Amazing Race: Robot Edition. In this task, the robot is placed in a real, unknown environment, without a map, and asked to find a designated location. It will need to explore its surroundings, find and approach people, engage them in a dialogue to obtain directions to the goal, and follow those directions to the hallway with the goal. We describe and implement a variety of robotic behaviors that performs each of these functions. We test these in the real world in test environments that were distinct from the training environments where we developed our methods and trained our models. Additionally, these test environments were completely unmodified and reflect the state of the real world.

First, we describe how our robotic system solves this problem where the environment is constrained to a single floor or a single building. We demonstrate that we are able to find a goal location in never-before-seen environments. Next, we describe a machine-learned approach to the dialogue and components of our system to make it more robust to the diversity and noisiness of navigational instructions someone may provide.