Quantifying and Developing Countermeasures for the Effect of Fatigue-Related Stressors on Automation Use and Trust During Robotic Supervisory Control

This project aims to develop and evaluate adaptive automation countermeasures to mitigate human performance decrements caused by sleep deprivation (SD) conditions when supervising high-autonomy robots and systems. We focus on problem solving and decision making tasks that are likely to require active human intervention and be impacted by SD. We also aim to increase our knowledge about the nature of performance and performance degradation on supervisory tasks when an individual is sleep-deprived.

During the second year, we conducted another experiment investigating the effects of SD on human performance during robotic supervisory control. We used a similar inpatient protocol in Experiment 2 as used in Experiment 1. We reused two task types developed in the first experiment: i) Efficiency Tasks - discover and use more efficient task order; and ii) Constraint Tasks - recall and use equipment status changes. Additionally, two countermeasures were developed to mitigate the effects of SD on human performance when supervising robots, and evaluated in the second experiment. The task planning countermeasure software was developed to aid short-term memory when sleep-deprived by allowing users to build and execute new sequences of automation tasks on-the-fly. This countermeasure should improve task accuracy and timing. The adaptive alerting countermeasure was developed to remind users to perform infrequent but important supervisory tasks when sleep-deprived. This countermeasure should reduce the time automation waits for human intervention and thus reduce overall task timing.

During Experiment 1 we experienced data loss due to unexpected variation in the way subjects interpreted and responded to instructions. Tasks needed to be sufficiently constrained that successful performance would likely generate a predictable stream of events, thus making scoring of the behavior streams feasible and results across subjects comparable. We redesigned core tasks from Experiment 1 for Experiment 2 and were overall successful in providing the intended, complex task sequences. The core tasks were executed as planned in Experiment 1 with essentially very low data loss. Data were collected from 8 participants in Experiment 2. We had expected 15 participants for this experiment. The number of participants was less than expected because a delayed project start reduced the amount of time we had to perform Experiment 2 and analyze data from it. We also experienced difficulty in finding subjects with the necessary education and computer literacy to serve as rough astronaut surrogates.

We report on results from the core tasks and from the tasks assessing use of the task planning countermeasure. The factors for core tasks were sleep status (rested versus sleep deprived) and repetition (first versus second exposure to the type of task), with assignment of rested-first or rested-second counterbalanced across users. We have an n of 8 with large individual differences, hence low sensitivity, and we have done no inferential statics. Rather, we looked for suggestive patterns in performance. For Core tasks, we looked for patterns suggesting impact of SD and repetition on flexible reasoning. For task planning countermeasure assessment, we looked for patterns concerning how planner use influenced performance. We did not analyze data from use of the adaptive alerting countermeasure, because of reduced time available to conduct Experiment 2 and analyze data from it.

Our ability to detect effects of SD or Trial is limited by small n and large individual differences. We found some evidence of effects of SD on performance for one core task. The most suggestive pattern was found on the accuracy of performing a Constraint Task. For this task, the subject should devise a sequence of tasks that connect batteries to solar panels while complying with a new safety constraint designating how many batteries can be connected to a solar panel. Four of the eight users showed sensitivity to the constraint at some point, two of these four respected the constraint on both trials. Two users, however, did well on the first trial (which was also rested) but did not respect the constraint later when they were sleep deprived. That is, they showed a decrement in performance when sleep deprived, even though that was their second exposure to this constraint task. We consider this pattern of performance to be consistent with the hypothesis that SD reduces the chance a user will correctly integrate information that requires modifying the way a user is doing the task. This is an intriguing pattern though our n is small, suggesting that further investigation may be warranted.

We assessed use of the task planning countermeasure by requiring users to work on two tasks at once, the context where we expected a benefit from being able to automatically run a sequence of procedures. The primary task was executing a series of robot procedures automatically and the secondary task was manually identifying the most efficient paths for a Rover using a diagram of possible paths. The primary dependent variables were how much work could be done on these tasks in 12 minutes. We looked at data from three occurrences of this task. Summarizing across these occurrences, users who were re-using their plans, or had planned outside the time window had some advantage in number of tasks completed over users who planned within the time window. Those in the Plan re-use task also had the lowest number of errors on the secondary task. For all tasks considered, slips are low, with little impact of plan use. Slips were defined as errors due to insertion of extra tasks, skipped tasks, unnecessary task repetition, or mis-ordered tasks. There may be a trend for slips to increase over the course of a session. A large amount of data was collected in this project, but very limited time was available to analyze these data. Further analyses of these data are merited.

More »