Skip Navigation
Flight Opportunities

Evaluation of the Biosleeve Gesture Control Interface for Telerobotics in Microgravity

Completed Technology Project

Project Introduction

Evaluation of the Biosleeve Gesture Control Interface for Telerobotics in Microgravity
For future missions with human-robot teams, the NASA Space Technology Roadmap highlights challenges in human-robot interfaces for dexterous telerobotic control, gesture control of robotic assistants, and telesupervision. To help address these challenges, JPL is developing a sleeve-based gesture recognition interface that could be embedded and worn in IVA and EVA suits. The JPL BioSleeve incorporates electromyography sensors (to measure muscle activity), inertial sensors (IMUs), and magnetic field sensors, to provide intuitive force and position control signals from natural arm, hand, and finger movements. The current prototype has successfully achieved >95% accuracy in recognition of 20 static hand gestures and 10 dynamic arm gestures in ground testing However, both capabilities are affected by the presence of gravity: the patterns of muscle activity change with respect to pose-dependent loading of the arm, and the gravity vector has been used with the magnetic field vector to predict arm pose. Because the near-term planned operational environment is on the International Space Station (ISS), we need to test the gesture recognition performance in the absence of gravity. The parabolic flight aircraft will allow testing in a microgravity environment, which will provide critical data on the performance of the interface, as well as give the team invaluable operational experience that may influence the design, development, and test program for the interface as it advances toward multiple flight opportunities Key objectives for the reduced gravity flight tests are: 1. Evaluate effects of the microgravity environment on gesture recognition performance, compared to performance in ground tests, after training both on the ground and during flight 2. Demonstrate BioSleeve interface utility in the microgravity environment by performing a gesture control/manipulation task To accomplish the first objective, an operator will don the BioSleeve interface prior to flight, then during flight will perform calibrations on a selected set of gestures to train the interface classifier software, and then be tested on the full gesture set. For the second objective, the subject will perform a simulated grappling task to assess gestural driving of the ISS Canadarm2 by an astronaut on EVA who is anchored to the arm’s end effector for station-keeping. Gesture recognition results will be displayed in real-time, and all data will be collected and stored for offline analyses. These microgravity tests should mature the BioSleeve technology from TRL 4 to TRL 5 or 6 – a prototype demonstration in a relevant space environment This work is significant because NASA and commercial space companies could benefit from BioSleeve gesture control technology for human-robot interactions in multiple applications, such as: natural hand gestures to direct surface rovers by an astronaut on EVA; intuitive telerobotic control for remote manipulation and for high degree-of-freedom robots such as Robonaut and Canadarm2 on ISS; and force control for power-assisted EVA gloves or exoskeletons. In the near-term, we plan to propose BioSleeve technology demonstrations aboard the ISS for gestural control of the Canadarm2 and SPHERES robot platforms. More »

Anticipated Benefits

Primary U.S. Work Locations and Key Partners

Technology Transitions

Project Library

Share this Project

Organizational Responsibility

Project Management

Project Duration

Technology Maturity (TRL)

Target Destination

Light bulb

Suggest an Edit

Recommend changes and additions to this project record.