The hazards of split-sample validation in hydrological model calibration

JOURNAL OF HYDROLOGY

卷:566

Article

Arsenault, Richard¹  Brissette, Francois¹  Martel, Jean-Luc¹ 

[1] Ecole Technol Super, Dept Construct Engn, 1100 Notre Dame Ouesy, Montreal, PQ H3C 1K3, Canada

关键词: Model validation;    Model calibration;    Model performance;    Hydrological modeling;    Split-sample testing;   

DOI  :  10.1016/j.jhydrol.2018.09.027

来源: Elsevier

【 摘 要 】

This paper investigates the issues related to the use of validation in hydrological model calibration. Traditionally, models are calibrated and then assessed on an independent period (split-sample) to determine their adequacy in simulating streamflow as compared to observations. In this study, two hydrological models and three North American catchments are used to evaluate the effects of using validation to assess the model parameters' robustness on the model's actual simulation capabilities and accuracy in simulating streamflow. The length of the calibration period is increased from 1 to 16 years, and for each case a large number of randomly selected combinations of years are used for calibration and for validation using the Nash-Sutcliffe Efficiency metric. The calibrated model is then run on an independent 8-year test-period to assess the model's actual performance in simulation mode in unknown conditions. The process is bootstrapped 30 times to ensure the robustness of the results. The tests pit the calibration/validation methods on increasing calibration period lengths against a full calibration on the entire available dataset. Results show that the calibration on the full dataset is the optimal strategy as it generates the most robust parameter sets, provides the best model accuracy on an independent testing period and does not require assumption-making on the modeler's part. The calibrated parameter sets for each test-case were evaluated using the relative bias and correlation metrics, which revealed that the method transfers well to these two other metrics. Results also demonstrate the pitfalls of the commonly used split-sampling strategy, where good parameter sets may be discarded due to model performance discrepancies between calibration and validation periods. The conclusions point to the need to use as many years as possible in the calibration step and to entirely disregard the validation aspect under certain conditions.

【 授权许可】

Free