Thermodynamics modeling of deep learning systems for a temperature based filter pruning technique

Frontiers in Physics

Physics

M. Lapenna¹  F. Faglioni²  R. Fioresi³ 

[1] DIFA, University of Bologna, Bologna, Italy;Dipartimento di Scienze Chimiche e Geologiche, University of Modena and Reggio Emilia, Modena, Italy;FaBiT, University of Bologna, Bologna, Italy;

关键词: deep learning;    thermodynamics;    machine learning;    condensed matter physics;    statistical mechanics;    molecular dynamics;   

DOI  :  10.3389/fphy.2023.1145156

 received in 2023-01-15, accepted in 2023-05-02,  发布年份 2023

来源: Frontiers

【 摘 要 】

We analyse the dynamics of convolutional filters’ parameters of a convolutional neural networks during and after training, via a thermodynamic analogy which allows for a sound definition of temperature. We show that removing high temperature filters has a minor effect on the performance of the model, while removing low temperature filters influences majorly both accuracy and loss decay. This result could be exploited to implement a temperature-based pruning technique for the filters and to determine efficiently the crucial filters for an effective learning.

【 授权许可】

Unknown   
Copyright © 2023 Lapenna, Faglioni and Fioresi.