Experience of a low-maintenance distributed data management system

20th International Conference on Computing in High Energy and Nuclear Physics

物理学;计算机科学

Takase, Wataru^1 ; Matsumoto, Yoshimi^1 ; Hasan, Adil^2 ; Lodovico, Francesca Di^3 ; Watase, Yoshiyuki^1 ; Sasaki, Takashi^1

High Energy Accelerator Research Organization (KEK), Japan^1

University of Liverpool, United Kingdom^2

Queen Mary, University of London, United Kingdom^3

关键词: Distributed data managements;    Large data volumes;    Low maintenance;    Management tool;    Production system;    Satellite system;    Scientific data;    Storage spaces;   

Others  :  https://iopscience.iop.org/article/10.1088/1742-6596/513/3/032095/pdf
DOI  :  10.1088/1742-6596/513/3/032095

学科分类：计算机科学（综合）

来源: IOP

【 摘 要 】

In this paper we report on the setup, deployment and operation of a low-maintenance, policy-driven distributed data management system for scientific data based on the integrated Rule Oriented Data System (iRODS). The system is located at KEK, Tsukuba, Japan with a satellite system at QMUL, London, UK. The system has been running stably in production for more than two years with minimal management overhead. The management tool that was developed to support the production system is also described. In addition we describe a simple XOR-based approach to file backup that reduces the amount of storage space consumed. In situations of large data volumes this approach can be of great benefit.