Deep Residual Learning for Nonlinear Regression

Entropy

Guokui Nian¹  Tiantian Yang²  Dongwei Chen²  Fei Hu³ 

[1] College of Earth Science, University of Chinese Academy of Sciences, Beijing 100049, China;School of Mathematical and Statistical Sciences, Clemson University, Clemson, SC 29641, USA;State Key Laboratory of Atmospheric Boundary Layer Physics and Atmospheric Chemistry, Institute of Atmospheric Physics, Chinese Academy of Sciences, Beijing 100029, China;

关键词: nonlinear regression;    nonlinear approximation;    deep residual learning;    neural network;   

DOI  :  10.3390/e22020193

来源: DOAJ

【 摘 要 】

Deep learning plays a key role in the recent developments of machine learning. This paper develops a deep residual neural network (ResNet) for the regression of nonlinear functions. Convolutional layers and pooling layers are replaced by fully connected layers in the residual block. To evaluate the new regression model, we train and test neural networks with different depths and widths on simulated data, and we find the optimal parameters. We perform multiple numerical tests of the optimal regression model on multiple simulated data, and the results show that the new regression model behaves well on simulated data. Comparisons are also made between the optimal residual regression and other linear as well as nonlinear approximation techniques, such as lasso regression, decision tree, and support vector machine. The optimal residual regression model has better approximation capacity compared to the other models. Finally, the residual regression is applied into the prediction of a relative humidity series in the real world. Our study indicates that the residual regression model is stable and applicable in practice.

【 授权许可】

Unknown