The ATLAS Eventlndex: data flow and inclusion of other metadata

17th International Workshop on Advanced Computing and Analysis Techniques in Physics Research

物理学;计算机科学

Barberis, D.^1 ; Zárate, S E Cárdenas^2 ; Favareto, A.^3 ; Casani, A Fernandez^4 ; Gallas, E.J.^5 ; Montoro, C Garcia^4 ; De La Hoz, S Gonzalez^4 ; Hrivnac, J.^6 ; Malon, D.^7 ; Prokoshin, F.^2 ; Salt, J.^4 ; Sanchez, J.^4 ; Toebbicke, R.^8 ; Yuan, R.^6

Universitá di Genova and INFN, Genova, Italy^1

Universidad Técnica Federico Santa Maria, Valparaiso, Chile^2

Universitá degli Studi and INFN, Genova, Italy^3

Instituto de Física Corpuscular, Valencia, Spain^4

Department of Physics, Oxford University, Oxford, United Kingdom^5

Laboratoire de l'Accélérateur Linéaire, Orsay, France^6

Argonne National Laboratory, Argonne, United States^7

CERN, France^8

关键词: Auxiliary information;    Data consistency;    Data management system;    Decision information;    Event identification;    Hadoop platforms;    Messaging system;    Production system;   

Others  :  https://iopscience.iop.org/article/10.1088/1742-6596/762/1/012028/pdf
DOI  :  10.1088/1742-6596/762/1/012028

学科分类：计算机科学（综合）

来源: IOP

【 摘 要 】

The ATLAS EventIndex is the catalogue of the event-related metadata for the information collected from the ATLAS detector. The basic unit of this information is the event record, containing the event identification parameters, pointers to the files containing this event as well as trigger decision information. The main use case for the EventIndex is event picking, as well as data consistency checks for large production campaigns. The EventIndex employs the Hadoop platform for data storage and handling, as well as a messaging system for the collection of information. The information for the EventIndex is collected both at Tier-0, when the data are first produced, and from the Grid, when various types of derived data are produced. The EventIndex uses various types of auxiliary information from other ATLAS sources for data collection and processing: trigger tables from the condition metadata database (COMA), dataset information from the data catalogue AMI and the Rucio data management system and information on production jobs from the ATLAS production system. The ATLAS production system is also used for the collection of event information from the Grid jobs. EventIndex developments started in 2012 and in the middle of 2015 the system was commissioned and started collecting event metadata, as a part of ATLAS Distributed Computing operations.