Center Innovation Fund: SSC CIF
Cloud Computing for Science Data Processing in Support of Emergency Response Project
Project Description
Cloud computing enables users to create virtual computers, each one with the optimal configuration of hardware and software for a job. The number of virtual computers can be increased to process large data sets or reduce processing time. Large scale scientific applications of the cloud, in many cases, are still in development. For example, in the event of an environmental crisis, such as the Deepwater Horizon oil spill, tornadoes, Mississippi River flooding, or a hurricane, up to date information is one of the most important commodities for decision makers. The volume of remote sensing data that is needed to be processed to accurately retrieve ocean properties from satellite measurements can easily exceed a terabyte, even for a small region such as the Mississippi Sound. Often, with current infrastructure, the time required to download, process and analyze the large volumes of remote sensing data, limits data processing capabilities to provide timely information to emergency responders. The use of a cloud computing platform, like NASA's Nebula, can help eliminate those barriers. NASA Nebula was developed as an open-source cloud computing platform to provide an easily quantifiable and improved alternative to building additional expensive data centers and to provide an easier way for NASA scientists and researchers to share large, complex data sets with external partners and the public. Nebula was designed as an Infrastructure-as-a-Service (IaaS) implementation that provided scalable computing and storage for science data and Web-based applications. Nebula IaaS allowed users to unilaterally provision, manage, and decommission computing capabilities (virtual machine instances, storage, etc.) on an as-needed basis through a Web interface or a set of command-line tools. This project demonstrated a novel way to conduct large scale scientific data processing utilizing NASA's cloud computer, Nebula. Remote sensing data from the Deepwater Horizon oil spill site was analyzed to assess changes in concentration of suspended sediments in the area surrounding the spill site. Software for processing time series of satellite remote sensing data was packaged together with a computer code that uses web services to download the data sets from a NASA data archive and distribution system. The new application package was able to be quickly deployed on a cloud computing platform when, and only for as long as, processing of the time series data is required to support emergency response. Fast network connection between the cloud system and the data archive enabled remote processing of the satellite data without the need for downloading the input data to a local computer system: only the output data products are transferred for further analysis. NASA was a pioneer in cloud computing by having established its own private cloud computing data center called Nebula in 2009 at the Ames Research Center (Ames). Nebula provided high-capacity computing and data storage services to NASA Centers, Mission Directorates, and external customers. In 2012, NASA shut down Nebula based on the results of a 5-month test that benchmarked Nebula's capabilities against those of Amazon and Microsoft. The test found that public clouds were more reliable and cost effective and offered much greater computing capacity and better IT support services than Nebula.
More »
Anticipated Benefits
Benefits to NASA funded missions include an increase in the accuracy of final products by incorporating more data. Similar techniques could also be used to provide faster and more comprehensive data analysis for test complex operations. In a traditional IT environment, it takes several months and usually hundreds of hours of labor by several different people to procure, set up, configure, and maintain new IT infrastructure. NASA must comply with a host of data security and privacy policies, which can sometimes create a challenge in finding a collaborative environment in which to share data with its outside partners. By utilizing cloud computing capabilities, users gain access to powerful IT resources months faster and with far less effort than before. Cloud computing saves hundreds of staff hours, allowing NASA scientists to focus on mission-critical activities instead of IT infrastructure requirements. Benefits to NASA unfunded missions and planned missions could potentially be a new data processing approach that could be utilized to enhance product accuracy, reduce infrastructure costs, and achieve results faster would be enabled. Cloud computing is a significant departure from the traditional IT infrastructure model. It does require technical training as well as a shift in how our teams and programs think about IT resources. This shift will require a change in how budgeting, procurement, workflow, and approach toward processing, storing, and accessing data is accomplished. Processing software and accessing data, storing data and utilizing applications developed for cloud computing can be deployed easily and economically and can enhance the use of large datasets, like satellite data, for commercial space industry use, that may not have the in-house resources to generate remote sensing products. Benefits to other government agencies/government managed data centers, include the ability to operate like the Internet through the process of enabling computing resources to be accessed and shared as virtual resources in a secure and scalable manner, include taking cloud computing usage of networks of large groups of servers typically running low-cost consumer PC technology with specialized connections to spread data-processing chores across them. The benefits of cloud computing is to apply traditional supercomputing, or high-performance computing power, normally used by military and research facilities, to perform tens of trillions of computations per second, for other applications such as data processing, to provide increased data storage and/or to computing power.
More »
Project Library
Primary U.S. Work Locations and Key Partners
| Organizations Performing Work | Role | Type | Location |
|---|---|---|---|
Stennis Space Center
(SSC)
|
Lead Organization | NASA Center | Stennis Space Center, Mississippi |
Primary U.S. Work Locations
-
Mississippi
Suggest an Edit
Recommend changes and additions to this project record.
This is a historic project that was completed before the creation of TechPort on October 1, 2012. Available data has been included. This record may contain less data than currently active projects.