Top

Water Resources Management

Published in:

25-07-2022

Framework for Hyperparameter Impact Analysis and Selection for Water Resources Feedforward Neural Network

Authors: Xuan Wang, Wenchong Tian, Zhenliang Liao

Published in: Water Resources Management | Issue 11/2022

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

The Feedforward Neural Network (FNN) is currently commonly used in problems involving water resources. However, the hyperparameters have received minimal attention: (i) Very little research has looked at how the hyperparameters affect FNN performance. (ii) Trial-and-error hyperparameter selection may have a significant degree of randomness. For the impact analysis and selection of hyperparameters, a model-based framework was proposed in this study. Through the framework, it was possible to (i) estimate how the hyperparameters affect model performance, offering theoretical direction for the hyperparameter selection, and (ii) select the hyperparameters based on the posterior hyperparameter distributions, with a certain degree of robustness. A case study of surface water quality prediction was carried out with 1,492,992 FNNs developed and trained using different hyperparameter sets. Theoretical inferences from the hyperparameter impact analysis were drawn, and a set of hyperparameters was chosen based on the obtained posterior hyperparameter distributions. The framework’s applicability in the research on water resources FNN for the impact analysis and selection of hyperparameters was demonstrated through the case study.

previous article Assessing the Vulnerability of California Water Utilities to Wildfires

next article Optimal Agricultural Water Allocation for the Sustainable Development of Surface and Groundwater Resources

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Available only for authorised users

Arhonditsis GB, Perhar G, Zhang W et al (2008) Addressing equifinality and uncertainty in eutrophication models. Water Resour Res. https://doi.org/10.1029/2007WR005862CrossRef

Azad A, Karami H, Farzin S et al (2019) Modeling river water quality parameters using modified adaptive neuro fuzzy inference system. Water Sci Eng 12:45–54. https://doi.org/10.1016/j.wse.2018.11.001CrossRef

Bansal S, Ganesan G (2019) Advanced evaluation methodology for water quality assessment using artificial neural network approach. Water Resour Manag 33:3127–3141. https://doi.org/10.1007/s11269-019-02289-6CrossRef

Beck MB (1987) Water quality modeling: a review of the analysis of uncertainty. Water Resour Res. https://doi.org/10.1029/WR023i008p01393CrossRef

Berkhahn S, Fuchs L, Neuweiler I (2019) An ensemble neural network model for real-time prediction of urban floods. J Hydrol 575:743–754. https://doi.org/10.1016/j.jhydrol.2019.05.066CrossRef

Cho H, Kim Y, Lee E et al (2020) Basic enhancement strategies when using Bayesian optimization for hyperparameter tuning of deep neural networks. IEEE Access 8:52588–52608. https://doi.org/10.1109/ACCESS.2020.2981072CrossRef

Das D, Chakraborty T, Majumder M, Bandyopadhyay TK (2022) Estimation of runoff under changed climatic scenario of a Meso scale river by neural network based gridded model approach. Water Resour Manag. https://doi.org/10.1007/s11269-022-03211-3CrossRef

Del Rio-Chanona EA, Cong X, Bradford E et al (2019) Review of advanced physical and data-driven models for dynamic bioprocess simulation: Case study of algae–bacteria consortium wastewater treatment. Biotechnol Bioeng 116:342–353. https://doi.org/10.1002/bit.26881CrossRef

Diez-Sierra J, del Jesus M (2020) Long-term rainfall prediction using atmospheric synoptic patterns in semi-arid climates with statistical and machine learning methods. J Hydrol 586:124789. https://doi.org/10.1016/j.jhydrol.2020.124789CrossRef

Goodfellow I, Bengio Y, Courville A (2016) Deep learning. The MIT Press

Greff K, Srivastava RK, Koutnik J et al (2017) LSTM: a search space odyssey. IEEE Trans Neural Networks Learn Syst 28:2222–2232. https://doi.org/10.1109/TNNLS.2016.2582924CrossRef

Humphrey GB, Maier HR, Wu W et al (2017) Improved validation framework and R-package for artificial neural network models. Environ Model Softw 92:82–106. https://doi.org/10.1016/j.envsoft.2017.01.023CrossRef

Hunter JM, Maier HR, Gibbs MS et al (2018) Framework for developing hybrid process-driven, artificial neural network and regression models for salinity prediction in river systems. Hydrol Earth Syst Sci 22:2987–3006. https://doi.org/10.5194/hess-22-2987-2018CrossRef

Kang G, Gao JZ, Xie G (2017) Data-driven water quality analysis and prediction: a survey. Proc - 3rd IEEE Int Conf Big Data Comput Serv Appl Big Data Service 2017, pp 224–232. https://doi.org/10.1109/BigDataService.2017.40

Kingma DP, Ba J (2014) Adam: a method for stochastic optimization

Kingston GB, Maier HR, Lambert MF (2008) Bayesian model selection applied to artificial neural networks used for water resources modeling. Water Resour Res 44:1–12. https://doi.org/10.1029/2007WR006155CrossRef

Li X, Zecchin AC, Maier HR (2015) Improving partial mutual information-based input variable selection by consideration of boundary issues associated with bandwidth estimation. Environ Model Softw 71:78–96. https://doi.org/10.1016/j.envsoft.2015.05.013CrossRef

Lindenschmidt KE, Fleischbein K, Baborowski M (2007) Structural uncertainty in a river water quality modelling system. Ecol Modell. https://doi.org/10.1016/j.ecolmodel.2007.01.004CrossRef

Liu T, Liu Y, Wang D et al (2019) Artificial neural network modeling on the prediction of mass transfer coefficient for ozone absorption in RPB. Chem Eng Res Des 152:38–47. https://doi.org/10.1016/j.cherd.2019.09.027CrossRef

Maier HR, Jain A, Dandy GC, Sudheer KP (2010) Methods used for the development of neural networks for the prediction of water resource variables in river systems: Current status and future directions. Environ Model Softw 25:891–909. https://doi.org/10.1016/j.envsoft.2010.02.003CrossRef

Molnar C (2019) Interpretable machine learning: a guide for making black box models explainable. Leanpub. https://doi.org/10.1007/s10290-014-0202-9CrossRef

Müller J, Park J, Sahu R et al (2020) Surrogate optimization of deep neural networks for groundwater predictions. J Glob Optim. https://doi.org/10.1007/s10898-020-00912-0CrossRef

O’Neill RV, Rust B (1979) Aggregation error in ecological models. Ecol Modell 7:91–105. https://doi.org/10.1016/0304-3800(79)90001-2CrossRef

Santos Finck J, Correa Pedrollo O (2021) Facing losses of telemetric signal in real time forecasting of water level using artificial neural networks. Water Resour Manag 35:1119–1133. https://doi.org/10.1007/s11269-021-02782-xCrossRef

Schmidhuber J (2015) Deep learning in neural networks: an overview. Neural Netw 61:85–117. https://doi.org/10.1016/J.NEUNET.2014.09.003CrossRef

Sharifi H, Roozbahani A, Hashemy Shahdany SM (2021) Evaluating the performance of agricultural water distribution systems using FIS, ANN and ANFIS intelligent models. Water Resour Manag 35:1797–1816. https://doi.org/10.1007/s11269-021-02810-wCrossRef

Shi B, Wang P, Jiang J, Liu R (2018) Applying high-frequency surrogate measurements and a wavelet-ANN model to provide early warnings of rapid surface water quality anomalies. Sci Total Environ 610–611:1390–1399. https://doi.org/10.1016/j.scitotenv.2017.08.232CrossRef

Tariq S, Loy-Benitez J, Nam KJ et al (2021) Transfer learning driven sequential forecasting and ventilation control of PM2.5 associated health risk levels in underground public facilities. J Hazard Mater 406:124753. https://doi.org/10.1016/j.jhazmat.2020.124753

Tian W, Liao Z, Wang X (2019) Transfer learning for neural network model in chlorophyll-a dynamics prediction. Environ Sci Pollut Res 26:29857–29871. https://doi.org/10.1007/s11356-019-06156-0CrossRef

Tian W, Liao Z, Zhang J (2017) An optimization of artificial neural network model for predicting chlorophyll dynamics. Ecol Modell 364:42–52. https://doi.org/10.1016/J.ECOLMODEL.2017.09.013CrossRef

Tziachris P, Aschonitis V, Chatzistathis T et al (2020) Comparing machine learning models and hybrid geostatistical methods using environmental and soil covariates for soil pH prediction. ISPRS Int J Geo-Information. https://doi.org/10.3390/ijgi9040276CrossRef

Valipour M, Banihabib ME, Behbahani SMR (2013) Comparison of the ARMA, ARIMA, and the autoregressive artificial neural network models in forecasting the monthly inflow of Dez dam reservoir. J Hydrol 476:433–441. https://doi.org/10.1016/J.JHYDROL.2012.11.017CrossRef

Vijay S, Kamaraj K (2021) Prediction of water quality index in drinking water distribution system using activation functions based ann. Water Resour Manag 35:535–553. https://doi.org/10.1007/s11269-020-02729-8CrossRef

Vijayashanthar V, Qiao J, Zhu Z et al (2018) Modeling fecal indicator bacteria in urban waterways using artificial neural networks. J Environ Eng (united States) 144:1–9. https://doi.org/10.1061/(ASCE)EE.1943-7870.0001377CrossRef

Wang X, Tian W, Liao Z (2021) Statistical comparison between SARIMA and ANN’s performance for surface water quality time series prediction. Environ Sci Pollut Res. https://doi.org/10.1007/s11356-021-13086-3CrossRef

Wu W, Dandy GC, Maier HR (2014) Protocol for developing ANN models and its application to the assessment of the quality of the ANN model development process in drinking water quality modelling. Environ Model Softw 54:108–127. https://doi.org/10.1016/j.envsoft.2013.12.016CrossRef

Xiong L, O’Connor KM (2008) An empirical method to improve the prediction limits of the GLUE methodology in rainfall–runoff modeling. J Hydrol 349:115–124. https://doi.org/10.1016/J.JHYDROL.2007.10.029CrossRef

Title: Framework for Hyperparameter Impact Analysis and Selection for Water Resources Feedforward Neural Network
Authors: Xuan Wang
Wenchong Tian
Zhenliang Liao
Publication date: 25-07-2022
Publisher: Springer Netherlands
Published in: Water Resources Management / Issue 11/2022
Print ISSN: 0920-4741
Electronic ISSN: 1573-1650
DOI: https://doi.org/10.1007/s11269-022-03248-4

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Technik"

Springer Professional "Wirtschaft+Technik"

Other articles of this Issue 11/2022

Comparative Evaluation of Water Resource Data Policy Inventories Towards the Improvement of East African Climate and Water Data Infrastructure

A Life Cycle Oriented Multi‑objective Optimal Maintenance of Water Distribution: Model and Application

Impacts of Receding of the Lakes Located in the Arid and Semi-arid Areas on the Coastal Groundwater: Integrated Modeling and Experimental Study

Digital Corporate Social Responsibility Reporting in the Water Industry

Dam-Break Flow on Mobile Bed Through an Idealized City: Numerical Study

Multi-Step-Ahead Monthly Streamflow Forecasting Using Convolutional Neural Networks