From distributed machine to distributed deep learning: a comprehensive survey

Journal of Big Data

Research

Zahra Yazdanparast¹  Mohammad Dehghani² 

[1] Tarbiat Modares University, Tehran, Iran;University of Tehran, Tehran, Iran;

关键词: Artificial intelligence;    Machine learning;    Distributed machine learning;    Distributed deep learning;    Ditributed reinforcement learning;    Data-parallelism;    Model-parallelism;   

DOI  :  10.1186/s40537-023-00829-x

 received in 2023-07-14, accepted in 2023-09-17,  发布年份 2023

来源: Springer

【 摘 要 】

Artificial intelligence has made remarkable progress in handling complex tasks, thanks to advances in hardware acceleration and machine learning algorithms. However, to acquire more accurate outcomes and solve more complex issues, algorithms should be trained with more data. Processing this huge amount of data could be time-consuming and require a great deal of computation. To address these issues, distributed machine learning has been proposed, which involves distributing the data and algorithm across several machines. There has been considerable effort put into developing distributed machine learning algorithms, and different methods have been proposed so far. We divide these algorithms in classification and clustering (traditional machine learning), deep learning and deep reinforcement learning groups. Distributed deep learning has gained more attention in recent years and most of the studies have focused on this approach. Therefore, we mostly concentrate on this category. Based on the investigation of the mentioned algorithms, we highlighted the limitations that should be addressed in future research.

【 授权许可】

CC BY   
© Springer Nature Switzerland AG 2023

【 图 表 】

Fig. 3

Fig. 7

Fig. 1