A Hybrid MapReduce Implementation of PCA on Tianhe-2

Journal of Physics: Conference Series

Wei Yu^1¹  Yutong Lu^1,2,3²  Yili Qu^2³ 

[1]College of Computer, National University of Defense Technology, Changsha 410073, China^1

[2]National Supercomputer Center in Guangzhou, Guangzhou 510006, China^3

[3]School of Data and Computer Science, Sun Yat-Sen University, Guangzhou 510006, China^2

DOI  :  10.1088/1742-6596/1168/5/052013

学科分类：物理（综合）

来源: Institute of Physics Publishing Ltd.

【 摘 要 】

"Big Data" has been a popular word anywhere. Researchers want the data processing more efficient. PCA algorithm is an effective data reduction algorithm applied to almost all big data fields. Meanwhile, there are many Machine Learning Algorithm Library applied to provide commonly-used algorithm, but these algorithms do not make good use of the resources of the supercomputer system. This paper uses MapReduce Model to design and implement PCA algorithm using MPI + OpenMP + SIMD hybrid accelerator programming tools on Tianhe-2 and get a significant speedup.

【 授权许可】

Unknown