Scientific data analysis on data-parallel platforms.

Ulmer, Craig D. ; Bayer, Gregory W. ; Choe, Yung Ryn ; Roe, Diana C.

关键词: ALGORITHMS;    APPLIANCES;    DATA ANALYSIS;    PERFORMANCE;    STORAGE;   

DOI  :  10.2172/1011199
RP-ID  :  SAND2010-7471
PID  :  OSTI ID: 1011199
Others  :  TRN: US201109%%398

学科分类：社会科学、人文和艺术（综合）

美国|英语

来源: SciTech Connect

【 摘 要 】

As scientific computing users migrate to petaflop platforms that promise to generate multi-terabyte datasets, there is a growing need in the community to be able to embed sophisticated analysis algorithms in the computing platforms' storage systems. Data Warehouse Appliances (DWAs) are attractive for this work, due to their ability to store and process massive datasets efficiently. While DWAs have been utilized effectively in data-mining and informatics applications, they remain largely unproven in scientific workloads. In this paper we present our experiences in adapting two mesh analysis algorithms to function on five different DWA architectures: two Netezza database appliances, an XtremeData dbX database, a LexisNexis DAS, and multiple Hadoop MapReduce clusters. The main contribution of this work is insight into the differences between these DWAs from a user's perspective. In addition, we present performance measurements for ten DWA systems to help understand the impact of different architectural trade-offs in these systems.