Semi-Autonomous Telerobotic Manipulation for On-Orbit Spacecraft Servicing and Assembly over Time-Delayed Telemetry

As we deploy robotic systems into more unstructured real-world environments, the tasks which those robots are expected to perform grow very quickly in complexity. We demand a greater number of possible operations, more variable environmental conditions, and larger spaces of objects and materials which need to be manipulated. With these increases in complexity, however, come even greater increases in the number of potential task failure modes. When the cost of task failure is high, like in the case of surgery or on orbit robotic operations, effective and efficient task recovery is essential The levels of cognitive reasoning and scene understanding in these situations currently necessitates a human operator directly controlling the robotic platform. Direct teleoperation is appropriate for situations when the robot is sufficiently close to the human operator, but there are numerous contexts in which this is not the case. In space logistics and exploration, there are many complex tasks which we wish to be able to perform under unavoidable time delay between the human operator and remote robot This project aims to advance the state-of-the art in semi-autonomous telemanipulation under multisecond (2s – 10s) closed-loop delay. Such delays are commonly observed in sustained ground-to-Earth-orbit communication. This project also develops a methodology for gradually introducing and improving semi autonomous control in critical applications such as on-orbit telerobotics. Specifically, this work includes: 1) The development of a systems framework for semi-autonomous teleoperation through Intent-Prediction based Traded Control, 2) An evaluation of the usability and efficacy of that framework with an increasingly complex assembly task, and 3) A demonstration of how simple semi-autonomous behaviors can be used to bootstrap more advanced ones using machine learning. The first part of this work involved a pilot study of a simple semi-autonomous grasping task with real robotic hardware and network limitations imposed by the terrestrial internet. While we only collected data from a single untrained user, this pilot study provided invaluable insight into the practical considerations which make semi-autonomous teleoperation difficult. The second and third parts of this work included the development of a remote- and local-classification scheme for semi-autonomous structure assembly, respectively. The final version of this architecture is diagrammed in Figure 1. Each part reports the results from a multi-user study in which we evaluated the usability of the system, its potential to improve user performance, and its decrease user workload. In each study, we found our semi-autonomous architecture enabled significant performance improvements and workload reduction, but only the results from the second study were statistically significant. The fourth, and final part of this work included the integration of machine learning into the architecture in order to enable iterative improvements on the performance of the system. This part represents a "meta" loop-closure, where we show that we can bootstrap more complex and higher-performing semi-autonomous interfaces from simpler ones.