Effective Human-Robot Collaborative Work for Critical Missions

The goal of this research is to design effective human-robot interaction paradigms for engineers and astronauts participating in critical NASA missions. Specifically, this project focuses on improving human interactions with assistive, free-flying robots. Assistive free-flyers (AFFs), such as NASA’s Smart-SPHERES, are autonomous or semi-autonomous robots that have 6 degrees of freedom (DOF) for movement in three-dimensional space and can assist humans by supporting tasks in areas that are difficult to access or infeasible to instrument. We believe that AFFs hold great potential to improve the capabilities of NASA missions, for instance by collecting EVA/IVA data, mapping indoor and outdoor environments, serving as lightweight support for astronauts engaged in tasks with high cognitive demands, and acting as mobile surveillance cameras and telepresence platforms (Figure 1). Additionally, terrestrial AFFs, such as multirotors or robotic blimps, have the capability to aid humans on earth, such as by surveying construction sites and monitoring power lines. In this research, we examined critical challenges that must be addressed for AFFs to achieve their potential, such as supporting both proximal and distal operation of AFFs, developing AFF communication, signaling mechanisms, and social behaviors for proximal interactions, and developing knowledge regarding free-flyer acceptance and integration into workflows. Free-flying robots hold great potential due to their unique abilities to freely traverse and survey environments. However, little is known regarding how humans will actually interact with these robots and how they might be designed to successfully integrate into human environments. This research utilized a human-centric approach towards the design of free-flying robots, leading to new understandings regarding how to develop robots that can provide novel forms of beneficial assistance while engendering positive responses when operating in human environments. First, we developed a thorough understanding regarding the design space for human interaction with free-flying robots. We identified areas in which free flyer design may be informed by previous research into social robots, telepresence robots, and mobile ground robots, as well as interaction aspects requiring extensions to previous frameworks. For example, several assumptions, such as the presence of gravity, defined orientations, and the absence of movement when no commands are issued, that underlay previous work in mobile ground robots no longer hold for free flyers. Although certain findings, such as the effects of telepresence system height on social outcomes, may be adapted (e.g., height corresponding with flight altitude), other findings are difficult to apply due to the functional morphologies of modern free-flying robots. We took a task-based approach to identify critical aspects of free-flyer design, analyzing individual, commercial, and military use-cases for free-flyers in both terrestrial and space domains. Such use-cases include mobile sensing (e.g., environmental surveys, inventory and asset tracking), mediating human communication (e.g., telepresence platform, broadcasting system), light-weight task assistance (e.g., holding tools or scientific instruments), and domain applications (e.g., search-and-rescue, escort and guidance). As a result of this analysis, we developed three major design aspects for consideration when developing free flyers for distal or proximal interaction: control, expression, and ecological fit. Control, representing fundamental user input into the system, focuses on interaction from a user, who may be proximal or distal, to an AFF. Control design considerations include the form of the input scheme (e.g., the input channel, time and space concerns) as well as the representation and feedback the AFF provides (e.g., how to give users understandings regarding appropriate input, correcting breakdowns in communication). Expression represents communication from free flyers to users, both in terms of the underlying goals regarding the information the flyer desires to convey as well as the mechanisms through which a flyer can communicate. Ecological fit deals with how users will perceive and relate to free flyers at a high level, how free flyers will modify existing workplace dynamics, and general free flyer interaction paradigms. As a whole, these three aspects can be used to identify the specific interaction variables, such as flyer altitude, mediating control technology, or user mental models, which will define a user’s interaction (Figure 2). Our goal in developing this ontological understanding regarding the interactive AFF design space is to ground current and future studies within a comprehensive framework. The remainder of this research involved developing knowledge regarding the interaction variables identified within this framework. We conducted a series of studies within the space of free-flyer expression that examined how variables relating to robot flight motions, such as trajectories, velocities, and accelerations, as well as aspects of robot appearance and morphology, might be manipulated to improve human perceptions of robot safety and usability. The first study built theoretical knowledge regarding the connection between motion and perception and resulted in a model of robot motion primitives and a set of parameterized manipulations that can be applied to these primitives. The second study examined the design of luminescent signals for flying robots and showed how signals such as car blinker systems and human gaze might be applied to indicate flying robot intent. We also examined aspects of free-flyer control. We conducted a preliminary study to collect data on human behaviors that informed the design of three novel interfaces for remote users to interact with and task free-flyers to accomplish a variety of realistic tasks. We then conducted an in-person experiment that required participants to work with a free-flyer to accomplish tasks including environmental sound surveys, equipment inspection, air quality measurements, and tool location. In this experiment, an interface that provided participants with spatial support in the form of a three-dimensional map of the environment as well as planning and execution support in the form of an interactive timeline led to high performance in terms of objective measures of efficiency and bandwidth. These outcomes highlight the need for such support, which current free-flyer interfaces do not typically provide. Overall, this work aimed to further knowledge regarding the design of free-flying robots that are able to achieve user goals quickly and efficiently while acting in manner appropriate for human environments. This research made theoretical contributions by developing a design space that collects and synthesizes previous findings from fields including human-robot interaction, human computer interaction, and cognitive science that are relevant to the design of free-flying robots, while also identifying new areas for future study. In addition, this research builds theory regarding the relationships between variables within the design space, for example the demonstrating how flight motions can be augmented to better communicate robot intent and thus increase perceived safety and usability. This work also made methodological contributions by demonstrating a principled manner of conducting empirical studies to examine human interaction with free-flying robots. Finally, this work has a number of practical contributions for improving the design of free-flying robots, for instance showing validated methods for improving the communication of robot intent and the design of novel robot interfaces.