Sparse Restricted Boltzmann Machine Based on Data Class Entropy

2nd International Symposium on Application of Materials Science and Energy Materials

材料科学;能源学

Zhang, Guobin^1 ; Li, Xiaoli^1 ; Li, Xiaoguang^1

School of Liaoning University, Shenyang, Liaoning, China^1

关键词: Critical technique;    Curse of dimensionality;    Data class;    Estimation of distributions;    Hidden layers;    Overfitting;    Restricted boltzmann machine;    Sparse methods;   

Others  :  https://iopscience.iop.org/article/10.1088/1757-899X/490/4/042003/pdf
DOI  :  10.1088/1757-899X/490/4/042003

学科分类：材料科学（综合）

来源: IOP

【 摘 要 】

The sparse of RBM is a critical technique to prevent the over-fitting of neural networks. Current competition sparse methods suffered from the fix sparse threshold. To this point, an adaptive RBM sparse method based on the complexity of data class distribution is proposed to determine the sparse threshold adaptively. In the method, a hidden-layer based entropy computing is introduced to overcome the curse of dimensionality for the estimation of distribution. Experiments on MNIST and CIFAR-10 are conducted and show that the proposed method is more effective than CaRBM and GC-RBM.