Middleware for Processing Message Queues with Elasticity Support and Sequential Integrity of Asynchronous Message Processing

15th Brazilian Symposium on High Performance Computational Systems

计算机科学

Teixeira, Eduardo Henrique^1 ; De Araújo, Aletéia Patrícia Favacho^1

Department of Computer Science - CIC, University of Brasilia (UnB), Brasília, DF, Brazil^1

关键词: Classification criterion;    Data distribution;    Distributed applications;    Message processing;    Message queues;    On demands;    Threshold detection;   

Others  :  https://iopscience.iop.org/article/10.1088/1742-6596/649/1/012005/pdf
DOI  :  10.1088/1742-6596/649/1/012005

学科分类：计算机科学（综合）

来源: IOP

【 摘 要 】

Elasticity in computing refers to dynamically adjusting the amount of allocated resources to process a distributed application. In order to achieve this, mechanisms are needed to avoid the phenomenon of the elasticity threshold detection moving constantly up or down. The existing work fails to deliver sequential integrity of asynchronous messages processing and the asymmetries of data distribution to achieve parallel consumption. This paper fills this gaps and proposes a middleware solution to dynamically analyze the flow of message queue, and a mechanism to increase the parallelized consumption based on the output behavior. An architecture for IOD (Increase On Demand) middleware is presented, with support for the increase and decrease of thread's to cope with the growth of message queues, using the technique of limit-based heuristics over a given period of time and grouping messages into sub-queues based on classification criteria.