Distributing CMS Data between the Florida T2 and T3 Centers using Lustre and Xrootd-fs

20th International Conference on Computing in High Energy and Nuclear Physics

物理学;计算机科学

Kaganas, G.^1 ; Rodriguez, J.L.^1 ; Cheng, M.^1 ; Avery, P.^2 ; Bourilkov, D.^2 ; Fu, Y.^2 ; Palencia, J.^3

Florida International University, Miami

FL, United States^1

University of Florida, Gainesville

FL, United States^2

Pittsburgh Supercomputing Center, Pittsburgh

PA, United States^3

关键词: Kerberos authentication;    Large volumes;    Lustre file systems;    Private networks;    Remote data access;    Scalability test;    University of Florida;    Worker nodes;   

Others  :  https://iopscience.iop.org/article/10.1088/1742-6596/513/4/042037/pdf
DOI  :  10.1088/1742-6596/513/4/042037

学科分类：计算机科学（综合）

来源: IOP

【 摘 要 】

We have developed remote data access for large volumes of data over the Wide Area Network based on the Lustre filesystem and Kerberos authentication for security. In this paper we explore a prototype for two-step data access from worker nodes at Florida Tier3 centers, located behind a firewall and using a private network, to data hosted on the Lustre filesystem at the University of Florida CMS Tier2 center. At the Tier3 center we use a client which mounts securely the Lustre filesystem and hosts an XrootD server. The worker nodes access the data from the Tier3 client using POSIX compliant tools via the XrootD-fs filesystem. We perform scalability tests with up to 200 jobs running in parallel on the Tier3 worker nodes.