ATLAS Replica Management in Rucio: Replication Rules and Subscriptions

20th International Conference on Computing in High Energy and Nuclear Physics

物理学;计算机科学

Barisits, M.^1,2,3 ; Serfon, C.^3 ; Garonne, V.^3 ; Lassnig, M.^3 ; Stewart, G.^3 ; Beermann, T.^3 ; Vigne, R.^3 ; Goossens, L.^3 ; Nairz, A.^3 ; Molfetas, A.^4

University of Innsbruck, Innsbruck, Austria^1

Vienna University of Technology, Vienna, Austria^2

CERN, Geneva, Switzerland^3

University of Melbourne, Melbourne, Australia^4

关键词: Data generation;    Data management system;    Design and implementations;    Distributed data managements;    Performance objective;    Performance requirements;    Physics data;    Replica management;   

Others  :  https://iopscience.iop.org/article/10.1088/1742-6596/513/4/042003/pdf
DOI  :  10.1088/1742-6596/513/4/042003

学科分类：计算机科学（综合）

来源: IOP

【 摘 要 】

The ATLAS Distributed Data Management system stores more than 150PB of physics data across 120 sites globally. To cope with the anticipated ATLAS workload of the coming decade, Rucio, the next-generation data management system has been developed. Replica management, as one of the key aspects of the system, has to satisfy critical performance requirements in order to keep pace with the experiment's high rate of continual data generation. The challenge lies in meeting these performance objectives while still giving the system users and applications a powerful toolkit to control their data workflows. In this work we present the concept, design and implementation of the replica management in Rucio. We will specifically introduce the workflows behind replication rules, their formal language definition, weighting and site selection. Furthermore we will present the subscription component, which offers functionality for users to proclaim interest in data that has not been created yet. This contribution describes the concept and the architecture behind those components and will show the benefits made by this system.