Accelerating the image processing by the optimization strategy for deep learning algorithm DBN

EURASIP Journal on Wireless Communications and Networking

Changyan Ying¹  Zhen Huang¹  Changtian Ying² 

[1] School of Information Science and Engineering, Xinjiang University;School of Mechanical and Electrical Engineering, Shaoxing University;

关键词: Deep learning;    DBN;    Acceleration strategy;    Data skew;    RDD cache;   

DOI  :  10.1186/s13638-018-1255-6

来源: DOAJ

【 摘 要 】

Abstract In recent years, image processing especially for remote sensing technology has developed rapidly. In the field of remote sensing, the efficiency of processing remote sensing images has been a research hotspot in this field. However, the remote sensing data has some problems when processing by a distributed framework, such as Spark, and the key problems to improve execution efficiency are data skew and data reused. Therefore, in this paper, a parallel acceleration strategy based on a typical deep learning algorithm, deep belief network (DBN), is proposed to improve the execution efficiency of the DBN algorithm in Spark. First, the re-partition algorithm based on the tag set is proposed to the relief data skew problem. Second, the cache replacement algorithm on the basis of characteristics is proposed to automatic cache the frequently used resilient distributed dataset (RDD). By caching RDD, the re-computation time of frequently reused RDD is reduced, which lead to the decrease of total computation time of the job. The numerical and analysis verify the effectiveness of the strategy.

【 授权许可】

Unknown