BBNet: A Novel Convolutional Neural Network Structure in Edge-Cloud Collaborative Inference

Sensors

Chengwei Wang¹  Xin Ma¹  Hongbo Zhou¹  Haoran Yu¹  Weiwei Zhang¹ 

[1]College of Engineering, Huaqiao University, Quanzhou 362021, China

关键词: collaborative intelligence;    deep learning;    model compression;    feature compression;    cloud computing;   

DOI  :  10.3390/s21134494

来源: DOAJ

【 摘 要 】

Edge-cloud collaborative inference can significantly reduce the delay of a deep neural network (DNN) by dividing the network between mobile edge and cloud. However, the in-layer data size of DNN is usually larger than the original data, so the communication time to send intermediate data to the cloud will also increase end-to-end latency. To cope with these challenges, this paper proposes a novel convolutional neural network structure—BBNet—that accelerates collaborative inference from two levels: (1) through channel-pruning: reducing the number of calculations and parameters of the original network; (2) through compressing the feature map at the split point to further reduce the size of the data transmitted. In addition, This paper implemented the BBNet structure based on NVIDIA Nano and the server. Compared with the original network, BBNet’s FLOPs and parameter achieve up to 5.67× and 11.57× on the compression rate, respectively. In the best case, the feature compression layer can reach a bit-compression rate of 512×. Compared with the better bandwidth conditions, BBNet has a more obvious inference delay when the network conditions are poor. For example, when the upload bandwidth is only 20 kb/s, the end-to-end latency of BBNet is increased by 38.89× compared with the cloud-only approach.

【 授权许可】

Unknown