Resource management for data streaming applications

DFuse;Data fusion;Programming abstraction;Scheduling;Data streaming;Resource management;Streamline;Grid computing;Cloud computing

Agarwalla, Bikash Kumar ; Computing

University:Georgia Institute of Technology

Department:Computing

关键词: DFuse;    Data fusion;    Programming abstraction;    Scheduling;    Data streaming;    Resource management;    Streamline;    Grid computing;    Cloud computing;   

Others  :  https://smartech.gatech.edu/bitstream/1853/34836/1/agarwalla_bikash_k_201008_phd.pdf

美国|英语

来源: SMARTech Repository

【 摘 要 】

This dissertation investigates novel middleware mechanisms for building streamingapplications. Developing streaming applications is a challenging task because (i) they are continuous in nature; (ii) they require fusion of data coming from multiple sources to derive higher level information; (iii) they require efficient transport of data from/to distributed sources and sinks;(iv) they need access to heterogeneous resources spanning sensor networks and high performance computing; and (v) they are time critical in nature. My thesis is that an intuitive programming abstraction will make it easier to build dynamic, distributed, and ubiquitous data streaming applications. Moreover, such an abstraction will enable an efficient allocation of shared and heterogeneous computational resources thereby making it easier for domain experts to build these applications. In support of the thesis, I present a novel programming abstraction, called DFuse,that makes it easier to develop these applications. A domain expert only needs to specify the input and outputconnections to fusion channels, and the fusion functions. The subsystems developed in this dissertation take care of instantiating the application, allocating resources for the application (via the scheduling heuristic developed in this dissertation) and dynamicallymanaging the resources (via the dynamic scheduling algorithm presented in this dissertation). Through extensive performance evaluation, I demonstrate that the resources are allocated efficiently to optimize the throughput and latency constraints of an application.