Utility of collecting metadata to manage a large scale conditions database in ATLAS

20th International Conference on Computing in High Energy and Nuclear Physics

物理学;计算机科学

Gallas, E.J.^1 ; Albrand, S.^2 ; Borodin, M.^3 ; Formica, A.^4

Department of Physics, University of Oxford, Denys Wilkinson Building, Keble Road, Oxford

OX1 3RH, United Kingdom^1

Laboratoire de Physique Subatomique et Corpusculaire, Université Joseph Fourier Grenoble 1, CNRS/IN2P3, 53 avenue des Martyrs, Grenoble

38026, France^2

National Research Nuclear University MEPhI, Kashirskoe sh. 31, Moscow

115409, Russia^3

CEA/Saclay IRFU/SEDI, Gif-sur-Yvette

91191, France^4

关键词: Common infrastructures;    Conditions database;    Data reconstruction;    Global informations;    Information infrastructures;    Large scale conditions;    Metadata repositories;    Structural metadata;   

Others  :  https://iopscience.iop.org/article/10.1088/1742-6596/513/4/042020/pdf
DOI  :  10.1088/1742-6596/513/4/042020

学科分类：计算机科学（综合）

来源: IOP

【 摘 要 】

The ATLAS Conditions Database, based on the LCG Conditions Database infrastructure, contains a wide variety of information needed in online data taking and offline analysis. The total volume of ATLAS conditions data is in the multi-Terabyte range. Internally, the active data is divided into 65 separate schemas (each with hundreds of underlying tables) according to overall data taking type, detector subsystem, and whether the data is used offline or strictly online. While each schema has a common infrastructure, each schema's data is entirely independent of other schemas, except at the highest level, where sets of conditions from each subsystem are tagged globally for ATLAS event data reconstruction and reprocessing. The partitioned nature of the conditions infrastructure works well for most purposes, but metadata about each schema is problematic to collect in global tools from such a system because it is only accessible via LCG tools schema by schema. This makes it difficult to get an overview of all schemas, collect interesting and useful descriptive and structural metadata for the overall system, and connect it with other ATLAS systems. This type of global information is needed for time critical data preparation tasks for data processing and has become more critical as the system has grown in size and diversity. Therefore, a new system has been developed to collect metadata for the management of the ATLAS Conditions Database. The structure and implementation of this metadata repository will be described. In addition, we will report its usage since its inception during LHC Run 1, how it has been exploited in the process of conditions data evolution during LSI (the current LHC long shutdown) in preparation for Run 2, and long term plans to incorporate more of its information into future ATLAS Conditions Database tools and the overall ATLAS information infrastructure.