{"project":{"acronym":"","projectId":11542,"title":"Effective Human-Robot Collaborative Work for Critical Missions","primaryTaxonomyNodes":[{"taxonomyNodeId":10629,"taxonomyRootId":8816,"parentNodeId":10628,"level":3,"code":"TX04.4.1","title":"Multi-Modal and Proximate Interaction","definition":"Multi-modal interaction allows for humans to interact with robots using multiple modes of communication, e.g. voice, gesture recognition. Proximate interaction allows for humans to interact with a robot side-by-side. Technologies to assist in these can enable humans to safely and efficiently control a larger number of robotic and autonomous assets, reducing overall demands on astronauts’ time for future exploration missions.","exampleTechnologies":"Virtual environment (VE), multi-modal dialogue, robot-to-suit interfaces, intent recognition and reaction, feedback displays for proximate interaction","hasChildren":false,"hasInteriorContent":true}],"startTrl":2,"currentTrl":3,"endTrl":3,"benefits":"This research will also work towards improving the communication channel whereby robots convey information back to human partners. To this end, this research will leverage advances in social and cognitive psychology to imbue robots with the knowledge of and ability to use social cues. These cues, such as gestures and eye gaze, will enhance human-robot collaborative work regardless of task domain by creating more fluid and effective interactions. Overall, this research will make practical and appreciable gains to NASA's robotic systems by creating closed-loop, intuitive, minimal error human-robot interactions.","description":"The objective of this project is to improve human-robot interaction (HRI) in order to enhance the capability of NASA critical missions. This research will focus two top challenges identified by the NASA RTA panel: improving the understanding and expressing of intent between humans and robots and creating supervised autonomous HRI loops. To address these challenges, this proposal will explore the design space of inferring human intent and cognitive state and the design space of improving robot feedback. This proposal will utilize cutting-edge brain-computer interface (BCI) technology to aid robots in inferring user intent and cognitive state. Recent improvements in BCI allow for the continuous passive monitoring of human features such as attention and cognitive load via low-cost wireless headsets with no need for extra training. Robots armed with such knowledge will improve collaborative control by knowing when it is best to interrupt their human partners to request aid and gaining further contextual task-based knowledge. This information will be combined with other robotic sensors to increase robot understanding of human intent. This research will also work towards improving the communication channel whereby robots convey information back to human partners. To this end, this research will leverage advances in social and cognitive psychology to imbue robots with the knowledge of and ability to use social cues. These cues, such as gestures and eye gaze, will enhance human-robot collaborative work regardless of task domain by creating more fluid and effective interactions. Overall, this research will make practical and appreciable gains to NASA's robotic systems by creating closed-loop, intuitive, minimal error human-robot interactions.","startYear":2012,"startMonth":9,"endYear":2015,"endMonth":8,"statusDescription":"Completed","principalInvestigators":[{"contactId":43592,"canUserEdit":false,"firstName":"Bilge","lastName":"Mutlu","fullName":"Bilge Mutlu","fullNameInverted":"Mutlu, Bilge","primaryEmail":"bmutlu@wisc.edu","publicEmail":false,"nacontact":false}],"programDirectors":[{"contactId":84634,"canUserEdit":false,"firstName":"Claudia","lastName":"Meyer","fullName":"Claudia M Meyer","fullNameInverted":"Meyer, Claudia M","middleInitial":"M","primaryEmail":"claudia.m.meyer@nasa.gov","publicEmail":true,"nacontact":false}],"programExecutives":[{"contactId":84634,"canUserEdit":false,"firstName":"Claudia","lastName":"Meyer","fullName":"Claudia M Meyer","fullNameInverted":"Meyer, Claudia M","middleInitial":"M","primaryEmail":"claudia.m.meyer@nasa.gov","publicEmail":true,"nacontact":false}],"programManagers":[{"contactId":183514,"canUserEdit":false,"firstName":"Hung","lastName":"Nguyen","fullName":"Hung D Nguyen","fullNameInverted":"Nguyen, Hung D","middleInitial":"D","primaryEmail":"hung.d.nguyen@nasa.gov","publicEmail":true,"nacontact":false}],"projectManagers":[{"contactId":37,"canUserEdit":false,"firstName":"Terrence","lastName":"Fong","fullName":"Terrence W Fong","fullNameInverted":"Fong, Terrence W","middleInitial":"W","primaryEmail":"terry.fong@nasa.gov","publicEmail":true,"nacontact":false}],"coInvestigators":[{"contactId":99393,"canUserEdit":false,"firstName":"Daniel","lastName":"Szafir","fullName":"Daniel Szafir","fullNameInverted":"Szafir, Daniel","primaryEmail":"dszafir@cs.wisc.edu","publicEmail":false,"nacontact":false}],"website":"https://www.nasa.gov/directorates/spacetech/home/index.html","libraryItems":[{"caption":"Project Image Effective Human-Robot Collaborative Work for Critical Missions","file":{"fileExtension":"jpg","fileId":313891,"fileName":"11542-1363183949404","fileSize":194342,"objectId":306462,"objectType":{"lkuCodeId":889,"code":"LIBRARY_ITEMS","description":"Library Items","lkuCodeTypeId":182,"lkuCodeType":{"codeType":"OBJECT_TYPE","description":"Object Type"}},"objectTypeId":889,"fileSizeString":"189.8 KB"},"files":[{"fileExtension":"jpg","fileId":313891,"fileName":"11542-1363183949404","fileSize":194342,"objectId":306462,"objectType":{"lkuCodeId":889,"code":"LIBRARY_ITEMS","description":"Library Items","lkuCodeTypeId":182,"lkuCodeType":{"codeType":"OBJECT_TYPE","description":"Object Type"}},"objectTypeId":889,"fileSizeString":"189.8 KB"}],"id":306462,"title":"11542-1363183949404.jpg","description":"Project Image Effective Human-Robot Collaborative Work for Critical Missions","libraryItemTypeId":1095,"projectId":11542,"primary":true,"publishedDateString":"","contentType":{"lkuCodeId":1095,"code":"IMAGE","description":"Image","lkuCodeTypeId":341,"lkuCodeType":{"codeType":"LIBRARY_ITEM_TYPE","description":"Library Item Type"}}}],"transitions":[{"transitionId":75627,"projectId":11542,"transitionDate":"2015-08-01","path":"Closed Out","details":"The goal of this research is to design effective human-robot interaction paradigms for engineers and astronauts participating in critical NASA missions. Specifically, this project focuses on improving human interactions with assistive, free-flying robots. Assistive free-flyers (AFFs), such as NASA’s Smart-SPHERES, are autonomous or semi-autonomous robots that have 6 degrees of freedom (DOF) for movement in three-dimensional space and can assist humans by supporting tasks in areas that are difficult to access or infeasible to instrument. We believe that AFFs hold great potential to improve the capabilities of NASA missions, for instance by collecting EVA/IVA data, mapping indoor and outdoor environments, serving as lightweight support for astronauts engaged in tasks with high cognitive demands, and acting as mobile surveillance cameras and telepresence platforms (Figure 1). Additionally, terrestrial AFFs, such as multirotors or robotic blimps, have the capability to aid humans on earth, such as by surveying construction sites and monitoring power lines. In this research, we examined critical challenges that must be addressed for AFFs to achieve their potential, such as supporting both proximal and distal operation of AFFs, developing AFF communication, signaling mechanisms, and social behaviors for proximal interactions, and developing knowledge regarding free-flyer acceptance and integration into workflows. Free-flying robots hold great potential due to their unique abilities to freely traverse and survey environments. However, little is known regarding how humans will actually interact with these robots and how they might be designed to successfully integrate into human environments. This research utilized a human-centric approach towards the design of free-flying robots, leading to new understandings regarding how to develop robots that can provide novel forms of beneficial assistance while engendering positive responses when operating in human environments. First, we developed a thorough understanding regarding the design space for human interaction with free-flying robots. We identified areas in which free flyer design may be informed by previous research into social robots, telepresence robots, and mobile ground robots, as well as interaction aspects requiring extensions to previous frameworks. For example, several assumptions, such as the presence of gravity, defined orientations, and the absence of movement when no commands are issued, that underlay previous work in mobile ground robots no longer hold for free flyers. Although certain findings, such as the effects of telepresence system height on social outcomes, may be adapted (e.g., height corresponding with flight altitude), other findings are difficult to apply due to the functional morphologies of modern free-flying robots. We took a task-based approach to identify critical aspects of free-flyer design, analyzing individual, commercial, and military use-cases for free-flyers in both terrestrial and space domains. Such use-cases include mobile sensing (e.g., environmental surveys, inventory and asset tracking), mediating human communication (e.g., telepresence platform, broadcasting system), light-weight task assistance (e.g., holding tools or scientific instruments), and domain applications (e.g., search-and-rescue, escort and guidance). As a result of this analysis, we developed three major design aspects for consideration when developing free flyers for distal or proximal interaction: control, expression, and ecological fit. Control, representing fundamental user input into the system, focuses on interaction from a user, who may be proximal or distal, to an AFF. Control design considerations include the form of the input scheme (e.g., the input channel, time and space concerns) as well as the representation and feedback the AFF provides (e.g., how to give users understandings regarding appropriate input, correcting breakdowns in communication). Expression represents communication from free flyers to users, both in terms of the underlying goals regarding the information the flyer desires to convey as well as the mechanisms through which a flyer can communicate. Ecological fit deals with how users will perceive and relate to free flyers at a high level, how free flyers will modify existing workplace dynamics, and general free flyer interaction paradigms. As a whole, these three aspects can be used to identify the specific interaction variables, such as flyer altitude, mediating control technology, or user mental models, which will define a user’s interaction (Figure 2). Our goal in developing this ontological understanding regarding the interactive AFF design space is to ground current and future studies within a comprehensive framework. The remainder of this research involved developing knowledge regarding the interaction variables identified within this framework. We conducted a series of studies within the space of free-flyer expression that examined how variables relating to robot flight motions, such as trajectories, velocities, and accelerations, as well as aspects of robot appearance and morphology, might be manipulated to improve human perceptions of robot safety and usability. The first study built theoretical knowledge regarding the connection between motion and perception and resulted in a model of robot motion primitives and a set of parameterized manipulations that can be applied to these primitives. The second study examined the design of luminescent signals for flying robots and showed how signals such as car blinker systems and human gaze might be applied to indicate flying robot intent. We also examined aspects of free-flyer control. We conducted a preliminary study to collect data on human behaviors that informed the design of three novel interfaces for remote users to interact with and task free-flyers to accomplish a variety of realistic tasks. We then conducted an in-person experiment that required participants to work with a free-flyer to accomplish tasks including environmental sound surveys, equipment inspection, air quality measurements, and tool location. In this experiment, an interface that provided participants with spatial support in the form of a three-dimensional map of the environment as well as planning and execution support in the form of an interactive timeline led to high performance in terms of objective measures of efficiency and bandwidth. These outcomes highlight the need for such support, which current free-flyer interfaces do not typically provide. Overall, this work aimed to further knowledge regarding the design of free-flying robots that are able to achieve user goals quickly and efficiently while acting in manner appropriate for human environments. This research made theoretical contributions by developing a design space that collects and synthesizes previous findings from fields including human-robot interaction, human computer interaction, and cognitive science that are relevant to the design of free-flying robots, while also identifying new areas for future study. In addition, this research builds theory regarding the relationships between variables within the design space, for example the demonstrating how flight motions can be augmented to better communicate robot intent and thus increase perceived safety and usability. This work also made methodological contributions by demonstrating a principled manner of conducting empirical studies to examine human interaction with free-flying robots. Finally, this work has a number of practical contributions for improving the design of free-flying robots, for instance showing validated methods for improving the communication of robot intent and the design of novel robot interfaces.","infoText":"Closed out","infoTextExtra":"","dateText":"August 2015"}],"primaryImage":{"file":{"fileExtension":"jpg","fileId":313891,"fileSizeString":"0 Byte"},"id":306462,"description":"Project Image Effective Human-Robot Collaborative Work for Critical Missions","projectId":11542,"publishedDateString":""},"responsibleMd":{"acronym":"STMD","canUserEdit":false,"city":"","external":false,"linkCount":0,"organizationId":4875,"organizationName":"Space Technology Mission Directorate","organizationType":"NASA_Mission_Directorate","naorganization":false,"organizationTypePretty":"NASA Mission Directorate"},"program":{"acronym":"STRG","active":true,"description":"
\tThe Space Technology Research Grants Program will accelerate the development of "push" technologies to support the future space science and exploration needs of NASA, other government agencies and the commercial space sector. Innovative efforts with high risk and high payoff will be encouraged. The program is composed of two competitively awarded components.
","programId":69,"responsibleMd":{"acronym":"STMD","canUserEdit":false,"city":"","external":false,"linkCount":0,"organizationId":4875,"organizationName":"Space Technology Mission Directorate","organizationType":"NASA_Mission_Directorate","naorganization":false,"organizationTypePretty":"NASA Mission Directorate"},"responsibleMdId":4875,"stockImageFileId":36658,"title":"Space Technology Research Grants"},"leadOrganization":{"canUserEdit":false,"city":"Madison","country":{"abbreviation":"US","countryId":236,"name":"United States"},"countryId":236,"external":true,"linkCount":0,"organizationId":4590,"organizationName":"University of Wisconsin-Madison","organizationType":"Academia","stateTerritory":{"abbreviation":"WI","country":{"abbreviation":"US","countryId":236,"name":"United States"},"countryId":236,"name":"Wisconsin","stateTerritoryId":32},"stateTerritoryId":32,"murepUnitId":240444,"naorganization":false,"organizationTypePretty":"Academia"},"supportingOrganizations":[{"acronym":"ARC","canUserEdit":false,"city":"Moffett Field","country":{"abbreviation":"US","countryId":236,"name":"United States"},"countryId":236,"external":false,"linkCount":0,"organizationId":4941,"organizationName":"Ames Research Center","organizationType":"NASA_Center","stateTerritory":{"abbreviation":"CA","country":{"abbreviation":"US","countryId":236,"name":"United States"},"countryId":236,"name":"California","stateTerritoryId":59},"stateTerritoryId":59,"naorganization":false,"organizationTypePretty":"NASA Center"}],"statesWithWork":[{"abbreviation":"WI","country":{"abbreviation":"US","countryId":236,"name":"United States"},"countryId":236,"name":"Wisconsin","stateTerritoryId":32}],"lastUpdated":"2024-2-6","releaseStatusString":"Released","viewCount":459,"endDateString":"Aug 2015","startDateString":"Sep 2012"}}