On the Disagreement of Forecasting Model Selection Criteria

Forecasting

article

Evangelos Spiliotis¹  Fotios Petropoulos²  Vassilios Assimakopoulos¹ 

[1] Forecasting and Strategy Unit, School of Electrical and Computer Engineering, National Technical University of Athens;School of Management, University of Bath

关键词: model selection;    information criteria;    time series;    exponential smoothing;    M4 competition;   

DOI  :  10.3390/forecast5020027

学科分类：陶瓷学

来源: mdpi

【 摘 要 】

Forecasters have been using various criteria to select the most appropriate model from a pool of candidate models. This includes measurements on the in-sample accuracy of the models, information criteria, and cross-validation, among others. Although the latter two options are generally preferred due to their ability to tackle overfitting, in univariate time-series forecasting settings, limited work has been conducted to confirm their superiority. In this study, we compared such popular criteria for the case of the exponential smoothing family of models using a large data set of real series. Our results suggest that there is significant disagreement between the suggestions of the examined criteria and that, depending on the approach used, models of different complexity may be favored, with possible negative effects on the forecasting accuracy. Moreover, we find that simple in-sample error measures can effectively select forecasting models, especially when focused on the most recent observations in the series.

【 授权许可】

CC BY