Performance and Advanced Data Placement Techniques with Ceph's Distributed Storage System

17th International Workshop on Advanced Computing and Analysis Techniques in Physics Research

物理学;计算机科学

Poat, M.D.^1 ; Lauret, J.^1

Brookhaven National Laboratory, P.O. Box 5000, Upton

NY

11973-5000, United States^1

关键词: Agile solutions;    Back-bone network;    Distributed storage;    Distributed storage system;    Maximum through-put;    New technologies;    Object storages;    Online computing;   

Others  :  https://iopscience.iop.org/article/10.1088/1742-6596/762/1/012025/pdf
DOI  :  10.1088/1742-6596/762/1/012025

学科分类：计算机科学（综合）

来源: IOP

【 摘 要 】

The STAR online computing environment is a demanding concentrated multipurpose compute system with the objective to obtain maximum throughput with process concurrency. Motivation for extending the STAR online compute farm from a simple job processing tool for data taking, into a multipurpose resource equipped with a large storage system would lead any dedicated resources to become an extremely efficient and an attractive multi-purpose facility. To achieve this goal, our compute farm is using the Ceph distributed storage system which has proven to be an agile solution due to its effective POSIX interface and excelling its object storage with IO concurrency. With this we have taken our cluster one step further in terms of IO performance by investigating and leveraging new technologies and key features of Ceph. With the acquisition of a 10Gb backbone network we have ensured to eliminate the network as a limitation. With further acquisition of large fast drives (1TB SSDs) we will show how one can customize the data placement options Ceph has to offer such as primary affinity, mounting OSD journals on SSDs, and Cache Tiering along with non-Ceph related local disk caching techniques. We will discuss the latest performance results along with the expected results by using each technique. We hope this paper will serve the community's interest for the Ceph distributed storage solution.