File Access Optimization with the Lustre Filesystem at Florida CMS T2

21st International Conference on Computing in High Energy and Nuclear Physics

物理学;计算机科学

Avery, P.^1 ; Bourilkov, D.^1 ; Fu, Y.^1 ; Kim, B.^1

University of Florida, Gainesville

FL, United States^1

关键词: Access methods;    Access patterns;    Backend system;    Data access patterns;    File access;    File transfers;    Lustre file systems;    Worker nodes;   

Others  :  https://iopscience.iop.org/article/10.1088/1742-6596/664/4/042028/pdf
DOI  :  10.1088/1742-6596/664/4/042028

学科分类：计算机科学（综合）

来源: IOP

【 摘 要 】

The Florida CMS Tier2 center, one of the CMS Tier2 centers, has been using the Lustre filesystem for its data storage backend system since 2004. Recently, the data access pattern at our site has changed greatly due to various new access methods that include file transfers through the GridFTP servers, read access from the worker nodes, and the remote read access through the xrootd servers. In order to optimize the file access performance, we have to consider all the possible access patterns and each pattern needs to be studied separately. In this presentation, we report on our work to optimize file access with the Lustre filesystem at the Florida CMS T2 using an approach based on analyzing these access patterns.