Game Changing Augmented Reality Training and Assistance for Maintenance Repair

NASA's vision to support human spaceflight missions beyond Low Earth Orbit will require game changing operational toolsets for training and assistance for maintenance operations.  With time delays of more than 15 minutes for communications back to earth, astronauts will face the daunting task of operating and maintaining numerous systems that might unexpectedly break or may even be required to perform life-saving surgery without support from the ground operators. Complex assembly and maintenance tasks in space operational environments are excellent targets for Augmented Reality (AR) applications.  The need for good training and access to large amounts of documentation are conditions making the use of AR techniques most promising. AR Training and Assistant Systems (AR-TAS) serve as "just-in-time" performance support tools for improving mission timelines while reducing prior training requirements.

The proposed technology reduces crew reliance of ground and paper procedure support in deep-space human spaceflight missions by providing real-time, in situ task guidance, and reducing the training burden by providing all relevant information into the operator field of view. AR has the potential to assist crew in many nominal and off nominal operational procedures execution, thus enabling crew autonomous operation and reducing their dependency on ground support. The marker-less AR system that minimizes tagging logistics and registration is especially attractive for space operations.  This technology uses a camera and image processing technique to create a "point cloud" map to register the real-world objects, which then enables the crew to get overlay information directly in the crew's field of view through the AR glasses. Procedure steps and animated 3D overlays of the device being worked on can significantly minimize the maintenance operations difficulty. The project is proposed as a multiyear effort.  Three AR prototypes were developed in year one to assess and to demonstrate the feasibility of the technology in the context of electronic procedures and maintenance operations.  The second year objectives are to enhance the vision processing capability and to improve robustness of the year one prototypes.  If successful, the third year will focus on assessing the feasibility to fly the proposed system as an ISS DTO.