Evaluating non-relational storage technology for HEP metadata and meta-data catalog

17th International Workshop on Advanced Computing and Analysis Techniques in Physics Research

物理学;计算机科学

Grigorieva, M.A.^1 ; Golosova, M.V.^1 ; Gubin, M.Y.^2 ; Klimentov, A.A.^1,3 ; Osipova, V.V.^2 ; Ryabinkin, E.A.^1

National Research Centre, Kurchatov Institute, Moscow, Russia^1

National Research Tomsk Polytechnic University, Tomsk, Russia^2

Brookhaven National Laboratory, Upton

NY, United States^3

关键词: Aggregated datum;    Combined informations;    Distributed computing environment;    Distributed data managements;    Relational storage;    Scientific experiments;    Specialized software;    Workload management;   

Others  :  https://iopscience.iop.org/article/10.1088/1742-6596/762/1/012017/pdf
DOI  :  10.1088/1742-6596/762/1/012017

学科分类：计算机科学（综合）

来源: IOP

【 摘 要 】

Large-scale scientific experiments produce vast volumes of data. These data are stored, processed and analyzed in a distributed computing environment. The life cycle of experiment is managed by specialized software like Distributed Data Management and Workload Management Systems. In order to be interpreted and mined, experimental data must be accompanied by auxiliary metadata, which are recorded at each data processing step. Metadata describes scientific data and represent scientific objects or results of scientific experiments, allowing them to be shared by various applications, to be recorded in databases or published via Web. Processing and analysis of constantly growing volume of auxiliary metadata is a challenging task, not simpler than the management and processing of experimental data itself. Furthermore, metadata sources are often loosely coupled and potentially may lead to an end-user inconsistency in combined information queries. To aggregate and synthesize a range of primary metadata sources, and enhance them with flexible schema-less addition of aggregated data, we are developing the Data Knowledge Base architecture serving as the intelligence behind GUIs and APIs.