분산 스트림 처리 엔진에서의 선언형 질의 언어 지원 및 성능 향상 방법

분산 스트림 처리 엔진;선언형 질의 언어;연속 질의;질의 최적화;621

공과대학 컴퓨터공학부 ;

University:서울대학교 대학원

关键词: 분산 스트림 처리 엔진;    선언형 질의 언어;    연속 질의;    질의 최적화;    621;   

Others  :  http://s-space.snu.ac.kr/bitstream/10371/122646/1/000000131941.pdf

美国|英语

来源: Seoul National University Open Repository

【 摘 要 】

There had been lots of research on stream data processing and systems that support querying on data stream. With the advent of big-data era, the volume of stream data and the users;; need for a complex query on stream data has increased. Because of that lack of memory and slow throughput has become a problem in single node environment. To process such big-data stream, many companies developed a distributed stream processing engine (DSPE), but they still lack declarative language support to use on it. So we must program each element every times to process stream on DSPE. To solve this problem, we use CQL as a query language and re-defined their operator to use on DSPE. After that, we describe generating process of query execution plan on DSPE using these operators. We also propose WP-sharing method to reduce network cost and heuristic methods for enhancing generated query execution plan. We have experimented with simple query and TPC-H Q10 query, to show the scalability of query execution plan. The result showed that the message throughput increased almost linearly as the worker increases. Also proposed heuristic methods showed better message throughput, and WP-sharing method increased message throughput 20% on simple query and 13% on TPC-H Q10 query. This result shows that WP-sharing method reduces network cost and results in better performance.