CloudTurbine: Streaming Data via Cloud File Sharing

We propose a novel technology to leverage rapidly evolving cloud based infrastructure to improve time constrained situational awareness for real-time decision making. Our "CloudTurbine" innovation eliminates the distinction between files and streams. Streaming and static data have long been considered separately. Whereas streaming data protocols continue to fragment, a great unification of approach for static data has occurred. The paradigm for file-sharing services is simple: (1) put data in a local file folder, (2) it automatically shows up at other linked systems. Wouldn't it be nice if one could unify an approach for streaming data while leveraging the file-sharing cloud infrastructure? This is precisely what we propose. CloudTurbine is a streaming data interface to file-sharing and file-transport services such as Dropbox and FTP. It delegates much of the data transmittal, security, and server resources to the third-party service. It eliminates the distinction between files and streams, and enables a simple, cost effective new paradigm for streaming data middleware. Phase I demonstrated the features, utility, and performance of the prototype CloudTurbine to be very appropriate for a wide range of data-streaming applications. A significant performance issue is the inherent latency imposed by file bundling plus transfer time. Testing has proven this to be tractable, with low latency (2-5s) for several file sharing services, and very low latency (10-50ms) for protocols such as FTP and mapped drives. We propose an Open Source CloudTurbine web portal as a hub to provide this technology to NASA, scientific, and industrial communities. Following the legacy of the earlier-generation DataTurbine, we will vigorously move to establish a self-sustaining Open Source community which provides CloudTurbine access, development, services, spin-off products, and community support. More »