A comparison of forecasting models for the resource usage of MapReduce applications

NEUROCOMPUTING

卷:418

Article

Li, Yang Yuan¹  Tien Van Do^2,3  Nguyen, Hai T.⁴ 

[1] Xian SiYuan Univ, Xian, Peoples R China

[2] Baoji Univ Arts & Sci, Baoji, Peoples R China

[3] Budapest Univ Technol & Econ, Dept Networked Syst & Serv, Magyar Tudosok Korutja 2, H-1117 Budapest, Hungary

[4] Budapest Univ Technol & Econ, Balatonfured Student Res Grp, Budapest, Hungary

关键词: MapReduce application;    Resource usage parameters;    LSTM model;    Multiple linear regression model;   

DOI  :  10.1016/j.neucom.2020.07.059

来源: Elsevier

【 摘 要 】

In this paper, we construct forecasting models (multivariate long short-term memory recurrent neural networks and multiple linear regression) for the resource usage prediction of four MapReduce applica-tions and applications executed within the Spark framework. We have evaluated the impact of a sample size to prediction accuracy. Also, we propose a phase modelling approach for read/write-intensive applications. Our results show that models based on long short-term memory recurrent neural networks exhibit a higher accuracy than multiple linear regression models and the intensive characteristics of a resource are closely related to the prediction accuracy of forecasting models. We investigated the hyper parameter tuning of such models and showed that a randomly initialised, shallow, well-tuned network may outperform deeper models that use stacked autoencoder initialisation. Furthermore, multivariate long short-term memory recurrent neural network models are more sensitive to sample size than multiple linear regression models. We show that an LSTM model trained in a specific machine may be used to predict the resource usage in another machine. (C) 2020 Elsevier B.V. All rights reserved.

【 授权许可】

Free