Master/Slave assignment optimization for high performance computing in an EC2 cloud using MPI

Network Protocols and Algorithms

Sven Koschnicke¹  Niklas Paulsen¹  Florian Schatz¹  Manfred Schimmler¹ 

[1]Christian-Albrechts-Universität zu Kiel

关键词: Master/Slave selection;    High Performance Computing;    Cloud Computing;    MPI;    Communication Benchmark;   

DOI  :  10.5296/npa.v4i1.1290

学科分类：计算机应用

来源: Macrothink Institute, Inc.

【 摘 要 】

For high performance computing, cloud services offer highly scaleable infrastructures on demand. Without requiring a great deal of maintenance and financial resources, which a datacenter would need, it is possible to obtain a huge set of computing instances from, e.g., the Amazon Elastic Computing Cloud (Amazon EC2). The downsides of cloud computing are that the set of computing instances is arbitrary and that communication speed varies greartly and affects the running time of algorithms. We present a master/slave selection algorithm for the EC2 cloud service based on a benchmark to counteract this disadvantage. The benchmark measures communication speed and the variance of communication in a NxN network consisting of N cloud instances. With the results of the benchmark, we perform a master/slave selection algorithm to maximize the number of messages between master and slaves. The results show that our proposed method greatly increases efficiency.

【 授权许可】

Unknown