DDM Workload Emulation

20th International Conference on Computing in High Energy and Nuclear Physics

物理学;计算机科学

Vigne, R.^1,2 ; Schikuta, E.^1 ; Garonne, V.^2 ; Stewart, G.^2 ; Barisits, M.^2 ; Beermann, T.^2 ; Lassnig, M.^2 ; Serfon, C.^2 ; Goossens, L.^2 ; Nairz, A.^2

University of Vienna, Austria^1

CERN, Geneva, Switzerland^2

关键词: ATLAS experiment;    Computing resource;    Data provisioning;    Distributed data managements;    Emulation framework;    Maintenance cost;    Stress Testing;   

Others  :  https://iopscience.iop.org/article/10.1088/1742-6596/513/4/042048/pdf
DOI  :  10.1088/1742-6596/513/4/042048

学科分类：计算机科学（综合）

来源: IOP

【 摘 要 】

Rucio is the successor of the current Don Quijote 2 (DQ2) system for the distributed data management (DDM) system of the ATLAS experiment. The reasons for replacing DQ2 are manifold, but besides high maintenance costs and architectural limitations, scalability concerns are on top of the list. Current expectations are that the amount of data will be three to four times as it is today by the end of 2014. Further is the availability of more powerful computing resources pushing additional pressure on the DDM system as it increases the demands on data provisioning. Although DQ2 is capable of handling the current workload, it is already at its limits. To ensure that Rucio will be up to the expected workload, a way to emulate it is needed. To do so, first the current workload, observed in DQ2, must be understood in order to scale it up to future expectations. The paper discusses how selected core concepts are applied to the workload of the experiment and how knowledge about the current workload is derived from various sources (e.g. analysing the central file catalogue logs). Finally a description of the implemented emulation framework, used for stress-testing Rucio, is given.