Comparison of sort algorithms in Hadoop and PCJ

Journal of Big Data

Marek Nowicki¹ 

[1] Faculty of Mathematics and Computer Science, Nicolaus Copernicus University in Toruń;

关键词: Parallel computing;    MapReduce;    Partitioned global address space;    PGAS;    Java;    PCJ;   

DOI  :  10.1186/s40537-020-00376-9

来源: DOAJ

【 摘 要 】

Abstract Sorting algorithms are among the most commonly used algorithms in computer science and modern software. Having efficient implementation of sorting is necessary for a wide spectrum of scientific applications. This paper describes the sorting algorithm written using the partitioned global address space (PGAS) model, implemented using the Parallel Computing in Java (PCJ) library. The iterative implementation description is used to outline the possible performance issues and provide means to resolve them. The key idea of the implementation is to have an efficient building block that can be easily integrated into many application codes. This paper also presents the performance comparison of the PCJ implementation with the MapReduce approach, using Apache Hadoop TeraSort implementation. The comparison serves to show that the performance of the implementation is good enough, as the PCJ implementation shows similar efficiency to the Hadoop implementation.

【 授权许可】

Unknown