Monitoring and controlling ATLAS data management: The Rucio web user interface

21st International Conference on Computing in High Energy and Nuclear Physics

物理学;计算机科学

Lassnig, M.^1 ; Beermann, T.^1 ; Vigne, R.^3 ; Barisits, M.^1 ; Garonne, V.^2 ; Serfon, C.^1

ATLAS Data Processing, Physics Department, CERN, Genéve

1211-23, Switzerland^1

Department of Physics, University of Oslo, Oslo

0316, Norway^2

Institute for Astro- and Particle Physics, University of Innsbruck, Innsbruck

6020, Austria^3

关键词: Data management system;    Design Principles;    Hadoop MapReduce;    Massively parallels;    Monitoring and controlling;    User perspectives;    Web user interface;    Web-based interface;   

Others  :  https://iopscience.iop.org/article/10.1088/1742-6596/664/6/062028/pdf
DOI  :  10.1088/1742-6596/664/6/062028

学科分类：计算机科学（综合）

来源: IOP

【 摘 要 】

The monitoring and controlling interfaces of the previous data management system DQ2 followed the evolutionary requirements and needs of the ATLAS collaboration. The new data management system, Rucio, has put in place a redesigned web-based interface based upon the lessons learnt from DQ2, and the increased volume of managed information. This interface encompasses both a monitoring and controlling component, and allows easy integration for usergenerated views. The interface follows three design principles. First, the collection and storage of data from internal and external systems is asynchronous to reduce latency. This includes the use of technologies like ActiveMQ or Nagios. Second, analysis of the data into information is done massively parallel due to its volume, using a combined approach with an Oracle database and Hadoop MapReduce. Third, sharing of the information does not distinguish between human or programmatic access, making it easy to access selective parts of the information both in constrained frontends like web-browsers as well as remote services. This contribution will detail the reasons for these principles and the design choices taken. Additionally, the implementation, the interactions with external systems, and an evaluation of the system in production, both from a technological and user perspective, conclude this contribution.