A Lightweight, High-performance I/O Management Package for Data-intensive Computing

Wang, Jun

University of Nebraska--Lincoln

关键词: Management;    99 General And Miscellaneous//Mathematics, Computing, And Information Science;    Anl;    I/O, Data Intensive, High-Performance;    Reliability;   

DOI  :  10.2172/909861
RP-ID  :  DOE/ER/25687
RP-ID  :  FG02-05ER25687
RP-ID  :  909861

美国|英语

来源: UNT Digital Library

【 摘 要 】

File storage systems are playing an increasingly important role in high-performance computing as the performance gap between CPU and disk increases. It could take a long time to develop an entire system from scratch. Solutions will have to be built as extensions to existing systems. If new portable, customized software components are plugged into these systems, better sustained high I/O performance and higher scalability will be achieved, and the development cycle of next-generation of parallel file systems will be shortened. The overall research objective of this ECPI development plan aims to develop a lightweight, customized, high-performance I/O management package named LightI/O to extend and leverage current parallel file systems used by DOE. During this period, We have developed a novel component in LightI/O and prototype them into PVFS2, and evaluate the resultant prototype—extended PVFS2 system on data-intensive applications. The preliminary results indicate the extended PVFS2 delivers better performance and reliability to users. A strong collaborative effort between the PI at the University of Nebraska Lincoln and the DOE collaborators—Drs Rob Ross and Rajeev Thakur at Argonne National Laboratory who are leading the PVFS2 group makes the project more promising.