Federated data storage and management infrastructure

17th International Workshop on Advanced Computing and Analysis Techniques in Physics Research

物理学;计算机科学

Zarochentsev, A.^1 ; Kiryanov, A.^2,3 ; Klimentov, A.^3,4 ; Krasnopevtsev, D.^3,5 ; Hristov, P.^6

Saint-Petersburg State University, Russia^1

Petersburg Nuclear Physics Institute, Russia^2

National Research Centre, Kurchatov Institute, Russia^3

Brookhaven National Laboratory, United States^4

National Research Nuclear University MEPhI, Russia^5

European Centre for Nuclear Research' CERN, France^6

关键词: Data intensive science;    Heterogeneous computing;    Large Hadron collider LHC;    Management infrastructure;    Orders of magnitude;    Performance measurements;    Scientific exploration;    Underlying networks;   

Others  :  https://iopscience.iop.org/article/10.1088/1742-6596/762/1/012016/pdf
DOI  :  10.1088/1742-6596/762/1/012016

学科分类：计算机科学（综合）

来源: IOP

【 摘 要 】

The Large Hadron Collider (LHC)' operating at the international CERN Laboratory in Geneva, Switzerland, is leading Big Data driven scientific explorations. Experiments at the LHC explore the fundamental nature of matter and the basic forces that shape our universe. Computing models for the High Luminosity LHC era anticipate a growth of storage needs of at least orders of magnitude; it will require new approaches in data storage organization and data handling. In our project we address the fundamental problem of designing of architecture to integrate a distributed heterogeneous disk resources for LHC experiments and other data-intensive science applications and to provide access to data from heterogeneous computing facilities. We have prototyped a federated storage for Russian T1 and T2 centers located in Moscow, St.-Petersburg and Gatchina, as well as Russian / CERN federation. We have conducted extensive tests of underlying network infrastructure and storage endpoints with synthetic performance measurement tools as well as with HENP-specific workloads, including the ones running on supercomputing platform, cloud computing and Grid for ALICE and ATLAS experiments. We will present our current accomplishments with running LHC data analysis remotely and locally to demonstrate our ability to efficiently use federated data storage experiment wide within National Academic facilities for High Energy and Nuclear Physics as well as for other data-intensive science applications, such as bio-informatics.