MobilePrune: Neural Network Compression via ℓ0 Sparse Group Lasso on the Mobile System

Sensors

Zhiwen Cao¹  Kaikai Zhao²  Yijie Wang²  Pan Li³  Yubo Shao³  Zhehao Peng³  Jianzhu Ma⁴  Xingang Peng⁵ 

[1]Department of Computer Graphics, Purdue University, West Lafayette, IN 47907, USA

[2]Department of Computer Science, Indiana University at Bloomington, Bloomington, IN 47405, USA

[3]Department of Computer Science, Purdue University, West Lafayette, IN 47907, USA

[4]Institute for Artificial Intelligence, Peking University, Beijing 100871, China

[5]Institute for Interdisciplinary Information Sciences, Tsinghua University, Beijing 100190, China

关键词: mobile computing;    model compression;    pruning network;    deep learning;    convolutional neural network;   

DOI  :  10.3390/s22114081

来源: DOAJ

【 摘 要 】

It is hard to directly deploy deep learning models on today’s smartphones due to the substantial computational costs introduced by millions of parameters. To compress the model, we develop an ${\ell }_0$-based sparse group lasso model called MobilePrune which can generate extremely compact neural network models for both desktop and mobile platforms. We adopt group lasso penalty to enforce sparsity at the group level to benefit General Matrix Multiply (GEMM) and develop the very first algorithm that can optimize the ${\ell }_0$ norm in an exact manner and achieve the global convergence guarantee in the deep learning context. MobilePrune also allows complicated group structures to be applied on the group penalty (i.e., trees and overlapping groups) to suit DNN models with more complex architectures. Empirically, we observe the substantial reduction of compression ratio and computational costs for various popular deep learning models on multiple benchmark datasets compared to the state-of-the-art methods. More importantly, the compression models are deployed on the android system to confirm that our approach is able to achieve less response delay and battery consumption on mobile phones.

【 授权许可】

Unknown