UnisonFlow: A Software-Defined Coordination Mechanism for Message-Passing Communication and Computation

IEEE Access

Yoshiyuki Kido¹  Susumu Date¹  Shinji Shimojo¹  Hiroaki Yamanaka²  Eiji Kawai²  Keichi Takahashi³  Dashdavaa Khureltulga³ 

[1] Applied Information Systems Research Division, Cybermedia Center, Osaka University, Suita, Japan;ICT Testbed Research and Development Promotion Center, National Institute of Information and Communications Technology, Tokyo, Japan;Multimedia Engineering Department, Graduate School of Information Science and Technology, Osaka University, Suita, Japan;

关键词: Message passing interface;    software defined network;    openflow;    kernel assistance;    interconnects;   

DOI  :  10.1109/ACCESS.2018.2829532

来源: DOAJ

【 摘 要 】

Message passing interface (MPI) communication performance is becoming one of the key factors heavily affecting the total performance of data-intensive applications running on computer clusters. Our software-defined networking (SDN)-enhanced MPI improves the performance of communication over interconnects by integrating flexible and dynamic network controllability of SDN into MPI. We have demonstrated that the acceleration of individual MPI communication primitives is feasible through our past work on the SDN-enhanced MPI. However, real-world MPI applications have not benefited from such accelerated communication primitives through our research achievements to date, because each of the distinct network control algorithms designed for various MPI communication primitives cannot be activated and coordinated with the execution of the MPI application. Therefore, this paper proposes UnisonFlow, a software-defined coordination mechanism for the SDN-enhanced MPI that performs network control in synchronization with the execution of applications. An experiment conducted on a real-computer cluster verifies that the interconnect control can be successfully performed in synchronization with the execution of the application. Furthermore, the synchronization is performed with a low overhead and its performance penalty is practically negligible.

【 授权许可】

Unknown