An integrated solution for remote data access

21st International Conference on Computing in High Energy and Nuclear Physics

物理学;计算机科学

Sapunenko, Vladimir^1 ; D'Urso, Domenico^2,4 ; Dell'Agnello, Luca^1 ; Vagnoni, Vincenzo^3 ; Duranti, Matteo^4,5

INFN-CNAF, Viale Berti-Pichat 6/2, Bologna

40127, Italy^1

ASDC, Via del Politecnico, Rome

00133, Italy^2

INFN Sezione di Bologna, Via Irnerio 46, Bologna

40126, Italy^3

INFN Sez. Perugia, Via A. Pascoli, Perugia

06123, Italy^4

University of Perugia, Piazza Universita 1, Perugia

06100, Italy^5

关键词: Astro-particles;    Computing community;    E-infrastructures;    High-latency networks;    Integrated solutions;    Parallel file system;    Remote data access;    Storage manager;   

Others  :  https://iopscience.iop.org/article/10.1088/1742-6596/664/4/042047/pdf
DOI  :  10.1088/1742-6596/664/4/042047

学科分类：计算机科学（综合）

来源: IOP

【 摘 要 】

Data management constitutes one of the major challenges that a geographically- distributed e-Infrastructure has to face, especially when remote data access is involved. We discuss an integrated solution which enables transparent and efficient access to on-line and near-line data through high latency networks. The solution is based on the joint use of the General Parallel File System (GPFS) and of the Tivoli Storage Manager (TSM). Both products, developed by IBM, are well known and extensively used in the HEP computing community. Owing to a new feature introduced in GPFS 3.5, so-called Active File Management (AFM), the definition of a single, geographically-distributed namespace, characterised by automated data flow management between different locations, becomes possible. As a practical example, we present the implementation of AFM-based remote data access between two data centres located in Bologna and Rome, demonstrating the validity of the solution for the use case of the AMS experiment, an astro-particle experiment supported by the INFN CNAF data centre with the large disk space requirements (more than 1.5 PB).