The ATLAS Eventindex: an event catalogue for experiments collecting large amounts of data

20th International Conference on Computing in High Energy and Nuclear Physics

物理学;计算机科学

Barberis, D.^1 ; Cranshaw, J.^2 ; Dimitrov, G.^3 ; Favareto, A.^1 ; Casaní, Á Fernández^4 ; Hoz, S González De La^4 ; Hivná, J.^5 ; Malon, D.^2 ; Nowak, M.^6 ; Cairols, J. Salt^4 ; Sánchez, J.^4 ; Sorokoletov, R.^7 ; Zhang, Q.^2

Università di Genova, INFN, Genova, Italy^1

Argonne National Laboratory, Argonne

IL, United States^2

CERN, Geneva, Switzerland^3

Instituto de Fisica Corpuscular (IFIC), University of Valencia and CSIC, Valencia, Spain^4

LAL, Université Paris-Sud, CNRS/IN2P3, Orsay, France^5

Brookhaven National Laboratory, Upton

NY, United States^6

University of Texas at Arlington, United States, Arlington

TX, United States^7

关键词: Data storage systems;    Data storage technology;    High energy physics experiments;    Large amounts of data;    Multiple use-cases;    Performance tests;    Scientific experiments;    Search criterion;   

Others  :  https://iopscience.iop.org/article/10.1088/1742-6596/513/4/042002/pdf
DOI  :  10.1088/1742-6596/513/4/042002

学科分类：计算机科学（综合）

来源: IOP

【 摘 要 】

Modern scientific experiments collect vast amounts of data that must be catalogued to meet multiple use cases and search criteria. In particular, high-energy physics experiments currently in operation produce several billion events per year. A database with the references to the files including each event in every stage of processing is necessary in order to retrieve the selected events from data storage systems. The ATLAS EventIndex project is studying the best way to store the necessary information using modern data storage technologies (Hadoop, HBase etc.) that allow saving in memory key-value pairs and select the best tools to support this application from the point of view of performance, robustness and ease of use. This paper describes the initial design and performance tests and the project evolution towards deployment and operation during 2014.