Augmented Reality Electronic Procedures Development and Deep Machine Learning Technology Integration

The goal of this project is to explore the use of Augmented Reality (AR) and Deep Machine Learning (DML) technologies to support human exploration missions. Using the IRAD funding, the team has extended the current eProc (electronic procedures) development environment to integrate the AR specific authoring elements. These include adding the new representation language elements to identify an AR step, the engine communication layer with the AR device, the 3D polygon/CAD model of the target object to be recognized as well as the associated marker, and animations/cues to layer on top of the actual object during the procedure execution. Also, to integrate DML technology, the team developed tools that automate the DML training dataset development process by generating large sets of labeled images using computer graphics techniques. So, the DML based AR cues can be incorporated and improved the robustness of object recognition and tracking in a AR applications and ultimately improve AR user experience. The current TRL is at 4-5, where a system design has been completed, prototypes are being developed to assess feasiblity of the overall system performance.