Accelerating Scientific Analysis with SciDB

21st International Conference on Computing in High Energy and Nuclear Physics

物理学;计算机科学

Gerhardt, L.^1 ; Faham, C.H.^1 ; Yao, Y.^1

Lawrence Berkeley National Laboratory, 1 Cyclotron Road Mailstop, Berkeley

CA

94720, United States^1

关键词: Analytical database;    Commodity hardware;    Dark matter detectors;    Lawrence Berkeley National Laboratory;    Open sources;    Scientific analysis;    Structured data;    Very large arrays;   

Others  :  https://iopscience.iop.org/article/10.1088/1742-6596/664/7/072019/pdf
DOI  :  10.1088/1742-6596/664/7/072019

学科分类：计算机科学（综合）

来源: IOP

【 摘 要 】

SciDB is an open-source analytical database for scalable complex analytics on very large array or multi-structured data from a variety of sources, programmable from Python and R. It runs on HPC, commodity hardware grids, or in a cloud and can manage and analyze terabytes of array-structured data and do complex analytics in-database. We present an overall description of the SciDB framework and describe its implementation at NERSC at Lawrence Berkeley National Laboratory. A case study using SciDB to analyze data from the LUX dark matter detector is described and future plans for a large SciDB array at NERSC are described.