{"project":{"acronym":"","projectId":93425,"title":"Context-Sensitive Augmented Reality for Mission Operations","startTrl":3,"currentTrl":6,"endTrl":6,"benefits":"This research will be immediately useful to the NASA JSC Hybrid Reality Laboratory (HRL), which will use our software to guide users through their ISS models and train users on interacting with these models. There are also applications for augmented reality on-board ISS, including applications of the HoloLens and tablet browser to procedure execution. This research can have immediate application to ISS operations because there are several iPads already on ISS and a Microsoft HoloLens. We are also integrating with NASA?s Dynamic On-board Ubiquitous Graphics (DOUG) system to assist in training for EVA and ROBO operations. PRIDE is being evaluated for use by ground operators for the Resource Prospector robotic mission to the moon being jointly developed by NASA JSC and ARC. Finally, we will work with NASA's Human Research Program (HRP) to identify applications for this work including analog test environments.
TRACLabs is already selling PRIDE as a commercial product with a major oil field services company as a launch customer. PRIDE is being field-tested at several sites world-wide before deployment in actual operations in mid-2017. Augmented reality would immediately increase the effectiveness of the PRIDE software by providing hands-free assistance in performing complex and dangerous procedures. TRACLabs expects additional customers in the oil and gas industry will deploy PRIDE once it has been proven effective. A major chemical manufacturer is also performing a pilot project in their plants using PRIDE for complex operations. In these cases, a heads-up display like the HoloLens or the DAQRI could be very useful in field environments In all of these cases, we will offer augmented reality systems as an ``add-on'' to the existing PRIDE software we deliver. Thus, we can immediately move this research out into industry by leveraging our existing PRIDE user base.","description":"Current NASA missions to the International Space Station (ISS) are heavily dependent upon ground controllers to assist crew members in performing routine operations and maintenance as well as responses to off-nominal situations. Performing these procedures often requires close collaboration between ground controllers who have deep knowledge of the spacecraft's systems. This collaboration becomes more difficult in extended missions and crew members will need to have more autonomy. Augmented and virtual reality technology can help replace some of the guidance that ground controllers offer to crew members during procedure execution. Our proposed approach focuses on integrating augmented and virtual reality technologies into the same tools that mission operations uses today, allowing for augmented and virtual reality assistance to be built and updated by flight controllers and other mission specialists just as they do for their other work products. In Phase I, TRACLabs integrated its PRIDE electronic procedure platform with augmented and virtual reality systems such as the HTC Vive. In Phase II we will extend the PRIDE platform to augmented reality devices such as the Microsoft HoloLens. TRACLabs is also partnered with the Georgia Institute of Technology's Augmented Environments Laboratory who will develop algorithms that dynamically adjust the augmented reality cues (both their position, orientation, and scale, along with the specific graphical elements used) such that they align with the necessary parts of the world without obscuring important parts of the world.","startYear":2017,"startMonth":4,"endYear":2021,"endMonth":6,"statusDescription":"Completed","principalInvestigators":[{"contactId":512361,"canUserEdit":false,"firstName":"David","lastName":"Kortenkamp","fullName":"David Kortenkamp","fullNameInverted":"Kortenkamp, David","primaryEmail":"Korten@Traclabs.Com","publicEmail":true,"nacontact":false}],"programDirectors":[{"contactId":206378,"canUserEdit":false,"firstName":"Jason","lastName":"Kessler","fullName":"Jason L Kessler","fullNameInverted":"Kessler, Jason L","middleInitial":"L","primaryEmail":"jason.l.kessler@nasa.gov","publicEmail":true,"nacontact":false}],"programExecutives":[{"contactId":215154,"canUserEdit":false,"firstName":"Jennifer","lastName":"Gustetic","fullName":"Jennifer L Gustetic","fullNameInverted":"Gustetic, Jennifer L","middleInitial":"L","primaryEmail":"jennifer.l.gustetic@nasa.gov","publicEmail":true,"nacontact":false}],"programManagers":[{"contactId":62051,"canUserEdit":false,"firstName":"Carlos","lastName":"Torrez","fullName":"Carlos Torrez","fullNameInverted":"Torrez, Carlos","primaryEmail":"carlos.torrez@nasa.gov","publicEmail":true,"nacontact":false}],"projectManagers":[{"contactId":461333,"canUserEdit":false,"firstName":"Theresa","lastName":"Stanley","fullName":"Theresa M Stanley","fullNameInverted":"Stanley, Theresa M","middleInitial":"M","primaryEmail":"theresa.m.stanley@nasa.gov","publicEmail":true,"nacontact":false},{"contactId":299301,"canUserEdit":false,"firstName":"Lui","lastName":"Wang","fullName":"Lui Wang","fullNameInverted":"Wang, Lui","primaryEmail":"lui.wang-1@nasa.gov","publicEmail":true,"nacontact":false},{"contactId":263188,"canUserEdit":false,"firstName":"Kathryn","lastName":"Packard","fullName":"Kathryn B Packard","fullNameInverted":"Packard, Kathryn B","middleInitial":"B","primaryEmail":"kathryn.b.packard@outlook.com","publicEmail":true,"nacontact":false}],"website":"","libraryItems":[{"file":{"fileExtension":"pdf","fileId":303189,"fileName":"SBIR_2016_2_BC_Z5.01-7920","fileSize":117278,"objectId":299738,"objectType":{"lkuCodeId":889,"code":"LIBRARY_ITEMS","description":"Library Items","lkuCodeTypeId":182,"lkuCodeType":{"codeType":"OBJECT_TYPE","description":"Object Type"}},"objectTypeId":889,"fileSizeString":"114.5 KB"},"files":[{"fileExtension":"pdf","fileId":303189,"fileName":"SBIR_2016_2_BC_Z5.01-7920","fileSize":117278,"objectId":299738,"objectType":{"lkuCodeId":889,"code":"LIBRARY_ITEMS","description":"Library Items","lkuCodeTypeId":182,"lkuCodeType":{"codeType":"OBJECT_TYPE","description":"Object Type"}},"objectTypeId":889,"fileSizeString":"114.5 KB"}],"id":299738,"title":"Briefing Chart","description":"Context-Sensitive Augmented Reality for Mission Operations, Phase II Briefing Chart","libraryItemTypeId":1222,"projectId":93425,"primary":false,"publishedDateString":"","contentType":{"lkuCodeId":1222,"code":"DOCUMENT","description":"Document","lkuCodeTypeId":341,"lkuCodeType":{"codeType":"LIBRARY_ITEM_TYPE","description":"Library Item Type"}}},{"caption":"Context-Sensitive Augmented Reality for Mission Operations, Phase II Briefing Chart Image","file":{"fileExtension":"png","fileId":295010,"fileName":"SBIR_2016_2_BC_Z5.01-7920","fileSize":99843,"objectId":291533,"objectType":{"lkuCodeId":889,"code":"LIBRARY_ITEMS","description":"Library Items","lkuCodeTypeId":182,"lkuCodeType":{"codeType":"OBJECT_TYPE","description":"Object Type"}},"objectTypeId":889,"fileSizeString":"97.5 KB"},"files":[{"fileExtension":"png","fileId":295010,"fileName":"SBIR_2016_2_BC_Z5.01-7920","fileSize":99843,"objectId":291533,"objectType":{"lkuCodeId":889,"code":"LIBRARY_ITEMS","description":"Library Items","lkuCodeTypeId":182,"lkuCodeType":{"codeType":"OBJECT_TYPE","description":"Object Type"}},"objectTypeId":889,"fileSizeString":"97.5 KB"}],"id":291533,"title":"Briefing Chart Image","description":"Context-Sensitive Augmented Reality for Mission Operations, Phase II Briefing Chart Image","libraryItemTypeId":1095,"projectId":93425,"primary":false,"publishedDateString":"","contentType":{"lkuCodeId":1095,"code":"IMAGE","description":"Image","lkuCodeTypeId":341,"lkuCodeType":{"codeType":"LIBRARY_ITEM_TYPE","description":"Library Item Type"}}},{"caption":"Final Summary Chart Image","file":{"fileExtension":"png","fileId":292507,"fileName":"1625108732274","fileSize":403360,"objectId":289024,"objectType":{"lkuCodeId":889,"code":"LIBRARY_ITEMS","description":"Library Items","lkuCodeTypeId":182,"lkuCodeType":{"codeType":"OBJECT_TYPE","description":"Object Type"}},"objectTypeId":889,"fileSizeString":"393.9 KB"},"files":[{"fileExtension":"png","fileId":292507,"fileName":"1625108732274","fileSize":403360,"objectId":289024,"objectType":{"lkuCodeId":889,"code":"LIBRARY_ITEMS","description":"Library Items","lkuCodeTypeId":182,"lkuCodeType":{"codeType":"OBJECT_TYPE","description":"Object Type"}},"objectTypeId":889,"fileSizeString":"393.9 KB"}],"id":289024,"title":"Final Summary Chart Image","description":"Final Summary Chart Image","libraryItemTypeId":1095,"projectId":93425,"primary":true,"publishedDateString":"","contentType":{"lkuCodeId":1095,"code":"IMAGE","description":"Image","lkuCodeTypeId":341,"lkuCodeType":{"codeType":"LIBRARY_ITEM_TYPE","description":"Library Item Type"}}}],"transitions":[{"transitionId":69811,"projectId":93425,"partner":"Other","transitionDate":"2017-04-01","path":"Advanced From","relatedProjectId":89894,"relatedProject":{"acronym":"","projectId":89894,"title":"Context-Sensitive Augmented Reality for Mission Operations","startTrl":4,"currentTrl":5,"endTrl":5,"benefits":"This research can have immediate application to ISS operations because there are several iPads already on ISS and a Microsoft HoloLens on its way. We will use ISS procedures as test cases for this project. We will work closely with a variety of NASA personnel who are working on next generation procedure displays. We also see applications to robotic missions including control of NASA's R2 and R5 robots and robots on other planetary surfaces. PRIDE is being evaluated for use in ground operators for the Resource Prospector robotic mission to the moon being jointly developed by NASA JSC and ARC. Ground operations personnel are currently evaluating PRIDE and this technology would be able to assist them in their operations. We also have a close relationship with the Autonomous Mission Operations TOCA Autonomous Operations Project (AMO-TOCA)being tested on-board ISS. This could serve as a potential Phase~III testbed on ISS. Finally, we have connections to the Human Research Program (HRP) at NASA and will work with those personnel to identify applications, including analog test environments, for this work. We will meet with all of these individuals in Phase~I to determine their requirements and use cases for augmented reality.
TRACLabs is already selling PRIDE as a commercial product with oil field services company Baker Hughes as a launch customer. Baker Hughes is field-testing PRIDE at several sites world-wide before deployment in actual operations in mid-2016. PRIDE is proving automation assistance to drilling operations. Augmented reality would immediately increase the effectiveness of the PRIDE software in drilling operations by providing assistance in performing complex and dangerous procedures. TRACLabs expects additional customers in the oil and gas industry will deploy PRIDE once it has been proven effective by Baker Hughes. TRACLabs also sees application of this technology in the automotive manufacturing area. TRACLabs performed a small pilot project for automotive supplier Magna (second largest in the world with 285 manufacturing facilities and over 125,000 employees) on flexible robotic assembly. This was successful, and after a tour of several Magna manufacturing facilities in North America, TRACLabs personnel are negotiating a follow-on contract for research and development. Augmented reality would be used to assign personnel on the manufacturing floor in performing their tasks and validating their work. We expect other manufacturing companies to be interested as well. Sierra Nevada Corporation has also purchased PRIDE licenses for use in their Dream Chaser program, which was recently selected to deliver cargo to ISS.","description":"Current NASA missions to the International Space Station are heavily dependent upon ground controllers to assist crew members in performing routine operations and maintenance as well as responses to off-nominal situations. Standard operating procedures are at the heart of spacecraft operations, with almost 5000 procedures for ISS alone. Performing these procedures often requires close collaboration between ground controllers who have deep knowledge of the spacecraft's systems and crew members who have on-board situation awareness. This close collaboration will become more difficult in extended missions and crew members will need to have more autonomy. Augmented reality technology can help replace some of the guidance that ground controllers offer to crew members during procedure execution. Augmented reality can also provide continuous and just-in-time training opportunities during extended missions as well as entertainment and social connection opportunities. Context-sensitive augmented reality provides different support depending upon the on-board situation and ties directly to procedures, system data, daily plans, background information, and robotic assistants. TRACLabs has developed a procedure integrated development environment called PRIDE that is currently being used by NASA for ISS and Orion procedures. TRACLabs proposes to integrate augmented reality technologies into PRIDE in collaboration with the Georgia Tech Augmented Environmental Lab. In particular, Georgia Tech has developed an augmented realty capable web browser and Javascript framework that will complement the PRIDE web-based procedure execution system. These two industry-leading technologies will form the platform on which a suite of context-sensitive augmented reality applications can be quickly developed and deployed for a variety of NASA applications.","startYear":2016,"startMonth":6,"endYear":2016,"endMonth":12,"statusDescription":"Completed","website":"","program":{"acronym":"SBIR/STTR","active":true,"description":"
The NASA SBIR and STTR programs fund the research, development, and demonstration of innovative technologies that fulfill NASA needs as described in the annual Solicitations and have significant potential for successful commercialization. If you are a small business concern (SBC) with 500 or fewer employees or a non-profit RI such as a university or a research laboratory with ties to an SBC, then NASA encourages you to learn more about the SBIR and STTR programs as a potential source of seed funding for the development of your innovations.
The SBIR and STTR programs have 3 phases:
The SBIR and STTR Phase I contracts last for 6 months with a maximum funding of $125,000, and Phase II contracts last for 24 months with a maximum funding of $750,000 - $1.5 million.
Opportunity for Continued Technology Development Post-Phase II:
The NASA SBIR/STTR Program currently has in place two initiatives for supporting its small business partners past the basic Phase I and Phase II elements of the program that emphasize opportunities for commercialization. Specifically, the NASA SBIR/STTR Program has the Phase II Enhancement (Phase II-E) and Phase II eXpanded (Phase II-X) contract options.
Please review the links below to obtain more information on the SBIR/STTR programs.
Provides an overview of the SBIR and STTR programs as implemented by NASA
Provides access to the annual SBIR/STTR Solicitations containing detailed information on the program eligibility requirements, proposal instructions and research topics and subtopics
Schedule and links for the SBIR/STTR solicitations and selection announcements
Federal and non-Federal sources of assistance for small business
Search our complete archive of awarded project abstracts to learn about what NASA has funded
Still have questions? Visit the program FAQs
","programId":73,"responsibleMd":{"acronym":"STMD","canUserEdit":false,"city":"","external":false,"linkCount":0,"organizationId":4875,"organizationName":"Space Technology Mission Directorate","organizationType":"NASA_Mission_Directorate","naorganization":false,"organizationTypePretty":"NASA Mission Directorate"},"responsibleMdId":4875,"stockImageFileId":36648,"title":"Small Business Innovation Research/Small Business Tech Transfer"},"lastUpdated":"2024-1-10","releaseStatusString":"Released","viewCount":57,"endDateString":"Dec 2016","startDateString":"Jun 2016"},"infoText":"Advanced from another project within the program","infoTextExtra":"Another project within the program (Context-Sensitive Augmented Reality for Mission Operations)","dateText":"April 2017"},{"transitionId":69810,"projectId":93425,"transitionDate":"2021-06-01","path":"Closed Out","closeoutDocuments":[{"title":"Final Summary Chart","file":{"fileExtension":"pdf","fileId":308131,"fileName":"1625109279701","fileSize":545863,"objectId":69810,"objectType":{"lkuCodeId":1841,"code":"TRANSITION_FILES","description":"Transition Files","lkuCodeTypeId":182,"lkuCodeType":{"codeType":"OBJECT_TYPE","description":"Object Type"}},"fileSizeString":"533.1 KB"},"transitionId":69810,"fileId":308131}],"infoText":"Closed out","infoTextExtra":"","dateText":"June 2021"}],"primaryImage":{"file":{"fileExtension":"png","fileId":292507,"fileSizeString":"0 Byte"},"id":289024,"description":"Final Summary Chart Image","projectId":93425,"publishedDateString":""},"responsibleMd":{"acronym":"STMD","canUserEdit":false,"city":"","external":false,"linkCount":0,"organizationId":4875,"organizationName":"Space Technology Mission Directorate","organizationType":"NASA_Mission_Directorate","naorganization":false,"organizationTypePretty":"NASA Mission Directorate"},"program":{"acronym":"SBIR/STTR","active":true,"description":"The NASA SBIR and STTR programs fund the research, development, and demonstration of innovative technologies that fulfill NASA needs as described in the annual Solicitations and have significant potential for successful commercialization. If you are a small business concern (SBC) with 500 or fewer employees or a non-profit RI such as a university or a research laboratory with ties to an SBC, then NASA encourages you to learn more about the SBIR and STTR programs as a potential source of seed funding for the development of your innovations.
The SBIR and STTR programs have 3 phases:
The SBIR and STTR Phase I contracts last for 6 months with a maximum funding of $125,000, and Phase II contracts last for 24 months with a maximum funding of $750,000 - $1.5 million.
Opportunity for Continued Technology Development Post-Phase II:
The NASA SBIR/STTR Program currently has in place two initiatives for supporting its small business partners past the basic Phase I and Phase II elements of the program that emphasize opportunities for commercialization. Specifically, the NASA SBIR/STTR Program has the Phase II Enhancement (Phase II-E) and Phase II eXpanded (Phase II-X) contract options.
Please review the links below to obtain more information on the SBIR/STTR programs.
Provides an overview of the SBIR and STTR programs as implemented by NASA
Provides access to the annual SBIR/STTR Solicitations containing detailed information on the program eligibility requirements, proposal instructions and research topics and subtopics
Schedule and links for the SBIR/STTR solicitations and selection announcements
Federal and non-Federal sources of assistance for small business
Search our complete archive of awarded project abstracts to learn about what NASA has funded
Still have questions? Visit the program FAQs
","programId":73,"responsibleMd":{"acronym":"STMD","canUserEdit":false,"city":"","external":false,"linkCount":0,"organizationId":4875,"organizationName":"Space Technology Mission Directorate","organizationType":"NASA_Mission_Directorate","naorganization":false,"organizationTypePretty":"NASA Mission Directorate"},"responsibleMdId":4875,"stockImageFileId":36648,"title":"Small Business Innovation Research/Small Business Tech Transfer"},"leadOrganization":{"canUserEdit":false,"city":"Webster","congressionalDistrict":"Texas 21","country":{"abbreviation":"US","countryId":236,"name":"United States"},"countryId":236,"external":true,"linkCount":0,"organizationId":2615,"organizationName":"TRACLabs, Inc.","organizationType":"Industry","stateTerritory":{"abbreviation":"TX","country":{"abbreviation":"US","countryId":236,"name":"United States"},"countryId":236,"name":"Texas","stateTerritoryId":29},"stateTerritoryId":29,"ein":"043456264 ","dunsNumber":"193786014","uei":"CEL6Q7QSVHA7","naorganization":false,"organizationTypePretty":"Industry"},"supportingOrganizations":[{"acronym":"JSC","canUserEdit":false,"city":"Houston","country":{"abbreviation":"US","countryId":236,"name":"United States"},"countryId":236,"external":false,"linkCount":0,"organizationId":4853,"organizationName":"Johnson Space Center","organizationType":"NASA_Center","stateTerritory":{"abbreviation":"TX","country":{"abbreviation":"US","countryId":236,"name":"United States"},"countryId":236,"name":"Texas","stateTerritoryId":29},"stateTerritoryId":29,"naorganization":false,"organizationTypePretty":"NASA Center"}],"statesWithWork":[{"abbreviation":"TX","country":{"abbreviation":"US","countryId":236,"name":"United States"},"countryId":236,"name":"Texas","stateTerritoryId":29}],"lastUpdated":"2024-1-10","releaseStatusString":"Released","viewCount":77,"endDateString":"Jun 2021","startDateString":"Apr 2017"}}