Semantic Summarization for Context Aware Manipulation of Data

NASA's exploration and scientific missions will produce terabytes of information. As NASA enters a new phase of space exploration, managing large amounts of scientific and operational data will become even more challenging. Robots conducting planetary exploration will produce data for selection and preparation of exploration sites. Robots and space probes will collect scientific data to improve understanding of the solar system. Satellites in low Earth orbit will collect data for monitoring changes in the Earth's atmosphere and surface environment. Key challenges for all these missions are understanding and summarizing what data have been collected and using this knowledge to improve data access. TRACLabs and CMU propose to develop context aware image manipulation software for managing data collected remotely during NASA missions. This software will filter and search large image archives using the temporal and spatial characteristics of images, and the robotic, instrument, and environmental conditions when images were taken. It also will implement techniques for finding which images show a terrain feature specified by the user. In Phase II we will implement this software and evaluate its effectiveness for NASA missions. At the end of Phase II, context aware image manipulation software at TRL 5-6 will be delivered to NASA.

Effective manipulation of large amounts of data is a recurring need in a diverse set of applications. Military: The military is actively pursuing unmanned vehicles for surveillance and reconnaissance, which will produce large amounts of information. This will require integrating information from multiple sources. The proposed software can support operations personnel in finding and understanding data collected during such multi-vehicle operations. Chemical and Nuclear: The operation of chemical and nuclear plants produces data for tracking resources, monitoring system health, and diagnosing problems. The proposed software can be extended to help operators find and understand information relevant to ongoing operational situations. Health: Home instrumentation can improve independence of function for the aging and cognitively impaired by enabling human performance monitoring. The proposed software could be extended to support caregivers and clinicians in finding relevant data about client activities to evaluate the effectiveness of these aids in improving client independence of function.

The management and analysis of the large amounts of exploration and science data collected by robots, space probes, and satellites provide unique opportunities as well as significant challenges to NASA. The proposed software has been demonstrated to be useful for managing images collected by a robot during simulated lunar planetary operations. It should prove similarly useful for other NASA missions such as deep space probes and Earth observing satellites where large amounts of image data are collected. Effective management of such geospatial information collected during exploration and science operations requires support for multiple concurrent uses of that data, including the planning of data collection operations, the collection of data, the assessment of the quality and completeness of that data, and the analysis of data. These uses can be aided by the proposed software to improve human understanding of what data have been collected and how to access this data.