Popularity Prediction Tool for ATLAS Distributed Data Management

20th International Conference on Computing in High Energy and Nuclear Physics

物理学;计算机科学

Beermann, T.^1,2 ; Maettig, P.^1 ; Stewart, G.^2,3 ; Lassnig, M.^2 ; Garonne, V.^2 ; Barisits, M.^2 ; Vigne, R.^2 ; Serfon, C.^2 ; Goossens, L.^2 ; Nairz, A.^2 ; Molfetas, A.^2,4

University of Wuppertal, Wuppertal, Germany^1

CERN, Geneva 23

CH-1211, Switzerland^2

University of Glasgow, Glasgow

G12 8QQ, United Kingdom^3

University of Melbourne, Melbourne, Australia^4

关键词: Data distribution;    Data management system;    Distributed data managements;    Historical information;    Input parameter;    Popularity predictions;    Real workloads;    Storage systems;   

Others  :  https://iopscience.iop.org/article/10.1088/1742-6596/513/4/042004/pdf
DOI  :  10.1088/1742-6596/513/4/042004

学科分类：计算机科学（综合）

来源: IOP

【 摘 要 】

This paper describes a popularity prediction tool for data-intensive data management systems, such as ATLAS distributed data management (DDM). It is fed by the DDM popularity system, which produces historical reports about ATLAS data usage, providing information about files, datasets, users and sites where data was accessed. The tool described in this contribution uses this historical information to make a prediction about the future popularity of data. It finds trends in the usage of data using a set of neural networks and a set of input parameters and predicts the number of accesses in the near term future. This information can then be used in a second step to improve the distribution of replicas at sites, taking into account the cost of creating new replicas (bandwidth and load on the storage system) compared to gain of having new ones (faster access of data for analysis). To evaluate the benefit of the redistribution a grid simulator is introduced that is able replay real workload on different data distributions. This article describes the popularity prediction method and the simulator that is used to evaluate the redistribution.