Distributional Approximation of the Classification Accuracy and Gaussian Mixture Models for Deep Learning

Deep Neural Network;Gaussian Mixture Models;Classification Loss;Computer Engineering

Variani, EhsanJedynal, Bruno M ;

Johns Hopkins University

关键词: Deep Neural Network;    Gaussian Mixture Models;    Classification Loss;    Computer Engineering;   

Others  :  https://jscholarship.library.jhu.edu/bitstream/handle/1774.2/60558/VARIANI-DISSERTATION-2015.pdf?sequence=1&isAllowed=n

瑞士|英语

来源: JOHNS HOPKINS DSpace Repository

【 摘 要 】

While the Deep Neural Networks (DNNs) have led significant improvementboost in many recognition tasks, their inherit structure and trainingalgorithm has not significantly changed from traditinal Neural Networks.Sequence of linear and non-linear layers are trained together usingthe Back Propagation algorithm.The idea of deep learning can be compared to the idea of flow diagramsand pipelines in the engieering design. From this point of view, the deepnet componentsare not restricted to be simple linear or non-linear functions. Morecomplicated components can be designed for DNNs and jointly trainedwithin the deep structure. This thesis introducestwo of such components: the Gaussian Mixture Model (GMM) layer and thea novel Classification layer.Each neuron in the Gaussian Mixture Model layeroutputs a GMM likelihood. This provides many newpossibilities for deep learning. First, it provides a deep density estimationusing a mixture of Gaussian distributions. In addition, it brings the possibilityof having trainable non-linearities which have parameters trained throughthe coarse of training.The Classification layer preserves a simple idea of classificationobjective; finding parameters which minimize the probability of classificationerror. In the proposed two step process, the distribution of classificationaccuracy is first approximated and then used to compute the classification error.The classification parameters are then computed to minimize this approximatederror.Using the Central Limit Theorem, it is shownthat the classification layer parameters obey a closed form equation for thecase of binaryclassifciation. For the multicalss, the parameters are trained using theback propagation algorithm.Both layers are experimented in several recognition tasks such as speech, imageand diseases recognition. Competitive results are demonstrated using thesecomponents with state of the art recognition techniques in these tasks.Furthermore, significant part of this thesis discussed the theoreticalaspects of the proposed components.