MonitAR

We propose to develop MonitAR, an Augmented Reality (AR) system that provides procedure completion guidance to astronauts. MonitAR will replace guidance from mission control during periods of long time delay or when communication with Earth is not possible. Astronauts using AR glasses will receive feedback from MonitAR via visual cues as they progress through procedures on the spacecraft. The visual cues will be provided when MonitAR determines the astronaut is executing a task (a specific step in the procedure) that deviates from the current procedure. MonitAR will then guide the astronaut back to completing the task in a way that fits with the procedure. During execution, the current and upcoming tasks are proactively displayed to the astronaut in a readable form. The key innovation is to apply Case-Based Reasoning (CBR) to enable MonitAR to predict the task the astronaut is beginning to execute rather than recognize it when completed. This look-ahead capability enables guidance to be provided early enough to avoid procedure/task failure. Moreover, CBR takes advantage of the astronaut?s extensive training to capture how procedures/tasks are completed and, thereby, avoid a cumbersome and brittle modeling effort. Astronaut procedures will be represented as tasks in a plan using Action Notation Modeling Language (ANML), a planning language already being used to represent astronaut procedures. By representing the procedures as plans, the different ways a procedure can be correctly executed will be captured directly from the existing procedures. More »