Skip to main content
Top
Published in: Earth Science Informatics 2/2023

10-03-2023 | RESEARCH

A hybrid approach combining the multi-dimensional time series k-means algorithm and long short-term memory networks to predict the monthly water demand according to the uncertainty in the dataset

Authors: Azar Niknam, Hasan Khademi Zare, Hassan Hosseininasab, Ali Mostafaeipour

Published in: Earth Science Informatics | Issue 2/2023

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

An authentic water consumption forecast is an auxiliary tool to support the management of the water supply and demand in urban areas. Providing a highly accurate forecasting model depends a lot on the quality of the input data. Despite the advancement of technology, water consumption in some places is still recorded by operators, so its database usually has some approximate and incomplete data. For this reason, the methods used to predict the water demand should be able to handle the drawbacks caused by the uncertainty in the dataset. In this regard, a structured hybrid approach was designed to cluster the customers and predict their water demand according to the uncertainty in the dataset. First, a fuzzy-based algorithm consisting of Forward-Filling, Backward-Filling, and Mean methods was innovatively proposed to impute the missing data. Then, a multi-dimensional time series k-means clustering technique was developed to group the consumers based on their consumption behavior, for which the missing data were estimated with fuzzy numbers. Finally, one forecasting model inspired by Long Short-Term Memory (LSTM) networks was adjusted for each cluster to predict the monthly water demand using the lagged demand and the temperature. This approach was implemented on the water time series of the residential consumers in Yazd, Iran, from January 2011 to November 2020. Based on the performance evaluation in terms of the Root Mean Squared Error (RMSE), the proposed approach had an acceptable level of confidence to predict the water demand of all the clusters.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
go back to reference Aghabozorgi S, Ying Wah T, Herawan T, Jalab HA, Shaygan MA, Jalali A (2014) A hybrid algorithm for clustering of time series data based on affinity search technique. Sci World J 2014 Aghabozorgi S, Ying Wah T, Herawan T, Jalab HA, Shaygan MA, Jalali A (2014) A hybrid algorithm for clustering of time series data based on affinity search technique. Sci World J 2014
go back to reference Altunkaynak A, Nigussie TA (2017) Monthly water consumption prediction using season algorithm and wavelet transform–based models. J Water Resour Plan Manag 143(6):04017011CrossRef Altunkaynak A, Nigussie TA (2017) Monthly water consumption prediction using season algorithm and wavelet transform–based models. J Water Resour Plan Manag 143(6):04017011CrossRef
go back to reference Amiri M, Jensen R (2016) Missing data imputation using fuzzy-rough methods. Neurocomputing 205:152–164CrossRef Amiri M, Jensen R (2016) Missing data imputation using fuzzy-rough methods. Neurocomputing 205:152–164CrossRef
go back to reference Antunes A, Andrade-Campos A, Sardinha-Lourenço A, Oliveira M (2018) Short-term water demand forecasting using machine learning techniques. J Hydroinf 20(6):1343–1366CrossRef Antunes A, Andrade-Campos A, Sardinha-Lourenço A, Oliveira M (2018) Short-term water demand forecasting using machine learning techniques. J Hydroinf 20(6):1343–1366CrossRef
go back to reference Aristiawati K, Siswantining T, Sarwinda D, Soemartojo SM (2019). Missing values imputation based on fuzzy C-Means algorithm for classification of chronic obstructive pulmonary disease (COPD). In AIP Conference Proceedings (Vol. 2192, No. 1, p. 060003). AIP Publishing LLC Aristiawati K, Siswantining T, Sarwinda D, Soemartojo SM (2019). Missing values imputation based on fuzzy C-Means algorithm for classification of chronic obstructive pulmonary disease (COPD). In AIP Conference Proceedings (Vol. 2192, No. 1, p. 060003). AIP Publishing LLC
go back to reference Bata Mt, Carriveau R, Ting DS-K (2020). Short-term water demand forecasting using hybrid supervised and unsupervised machine learning model. Smart Water, 5, 1-18 Bata Mt, Carriveau R, Ting DS-K (2020). Short-term water demand forecasting using hybrid supervised and unsupervised machine learning model. Smart Water, 5, 1-18
go back to reference Bokde N, Beck MW, Álvarez FM, Kulat K (2018) A novel imputation methodology for time series based on pattern sequence forecasting. Pattern Recogn Lett 116:88–96CrossRef Bokde N, Beck MW, Álvarez FM, Kulat K (2018) A novel imputation methodology for time series based on pattern sequence forecasting. Pattern Recogn Lett 116:88–96CrossRef
go back to reference Candelieri A (2017) Clustering and support vector regression for water demand forecasting and anomaly detection. Water 9(3):224CrossRef Candelieri A (2017) Clustering and support vector regression for water demand forecasting and anomaly detection. Water 9(3):224CrossRef
go back to reference Candelieri A, Giordani I, Archetti F, Barkalov K, Meyerov I, Polovinkin A, Zolotykh N (2019) Tuning hyperparameters of a SVM-based water demand forecasting system through parallel global optimization. Comput Oper Res 106:202–209CrossRef Candelieri A, Giordani I, Archetti F, Barkalov K, Meyerov I, Polovinkin A, Zolotykh N (2019) Tuning hyperparameters of a SVM-based water demand forecasting system through parallel global optimization. Comput Oper Res 106:202–209CrossRef
go back to reference Chen SM (1994) Fuzzy system reliability analysis using fuzzy number arithmetic operations. Fuzzy Sets Syst 64(1):31–38CrossRef Chen SM (1994) Fuzzy system reliability analysis using fuzzy number arithmetic operations. Fuzzy Sets Syst 64(1):31–38CrossRef
go back to reference Du B, Zhou Q, Guo J, Guo S, Wang L (2021) Deep learning with long short-term memory neural networks combining wavelet transform and principal component analysis for daily urban water demand forecasting. Expert Syst Appl 171:114571CrossRef Du B, Zhou Q, Guo J, Guo S, Wang L (2021) Deep learning with long short-term memory neural networks combining wavelet transform and principal component analysis for daily urban water demand forecasting. Expert Syst Appl 171:114571CrossRef
go back to reference El-Bakry M, Ali F, El-Kilany A, Mazen S (2021) Fuzzy based techniques for handling missing values. Int J Adv Comput Sci Appl 12(3) El-Bakry M, Ali F, El-Kilany A, Mazen S (2021) Fuzzy based techniques for handling missing values. Int J Adv Comput Sci Appl 12(3)
go back to reference García Valverde D, Quevedo Casín JJ, Puig Cayuela V, Saludes Closa J (2015) Water demand estimation and outlier detection from smart meter data using classification and Big Data methods. In 2nd New Developments in IT & Water Conference, 8–10 Rotterdam (Holland) (pp. 1–8) García Valverde D, Quevedo Casín JJ, Puig Cayuela V, Saludes Closa J (2015) Water demand estimation and outlier detection from smart meter data using classification and Big Data methods. In 2nd New Developments in IT & Water Conference, 8–10 Rotterdam (Holland) (pp. 1–8)
go back to reference Gil A, Quartulli M, Olaizola IG, Sierra B (2020) Learning Optimal Time Series Combination and Pre-Processing by Smart Joins. Appl Sci 10(18):6346CrossRef Gil A, Quartulli M, Olaizola IG, Sierra B (2020) Learning Optimal Time Series Combination and Pre-Processing by Smart Joins. Appl Sci 10(18):6346CrossRef
go back to reference Giordano D, Mellia M, Cerquitelli T (2021) K-mdtsc: K-multi-dimensional time-series clustering algorithm. Electronics 10(10):1166CrossRef Giordano D, Mellia M, Cerquitelli T (2021) K-mdtsc: K-multi-dimensional time-series clustering algorithm. Electronics 10(10):1166CrossRef
go back to reference Hammond M, Chen AS, Batica J, Butler D, Djordjević S, Gourbesville P et al (2018) A new flood risk assessment framework for evaluating the effectiveness of policies to improve urban flood resilience. Urban Water Journal 15(5):427–436CrossRef Hammond M, Chen AS, Batica J, Butler D, Djordjević S, Gourbesville P et al (2018) A new flood risk assessment framework for evaluating the effectiveness of policies to improve urban flood resilience. Urban Water Journal 15(5):427–436CrossRef
go back to reference Herrera M, Torgo L, Izquierdo J, Pérez-García R (2010) Predictive models for forecasting hourly urban water demand. J Hydrol 387(1–2):141–150CrossRef Herrera M, Torgo L, Izquierdo J, Pérez-García R (2010) Predictive models for forecasting hourly urban water demand. J Hydrol 387(1–2):141–150CrossRef
go back to reference Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780CrossRef Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780CrossRef
go back to reference Hu P, Tong J, Wang J, Yang Y, de Oliveira Turci L (2019) A hybrid model based on CNN and Bi-LSTM for urban water demand prediction. In 2019 IEEE Congress on evolutionary computation (CEC) (pp. 1088-1094). IEEE Hu P, Tong J, Wang J, Yang Y, de Oliveira Turci L (2019) A hybrid model based on CNN and Bi-LSTM for urban water demand prediction. In 2019 IEEE Congress on evolutionary computation (CEC) (pp. 1088-1094). IEEE
go back to reference Jun S, Jung D, Lansey KE (2021) Comparison of imputation methods for end-user demands in water distribution systems. J Water Resour Plan Manag 147(12):04021080CrossRef Jun S, Jung D, Lansey KE (2021) Comparison of imputation methods for end-user demands in water distribution systems. J Water Resour Plan Manag 147(12):04021080CrossRef
go back to reference Kavitha V, Punithavalli M (2010). Clustering time series data stream-a literature survey. arXiv preprint arXiv:1005.4270 Kavitha V, Punithavalli M (2010). Clustering time series data stream-a literature survey. arXiv preprint arXiv:1005.4270
go back to reference Kim SH, Yang HJ, Ng KS (2011) Incremental expectation maximization principal component analysis for missing value imputation for coevolving EEG data. J Zhejiang Univ Sci 12(8):687–697CrossRef Kim SH, Yang HJ, Ng KS (2011) Incremental expectation maximization principal component analysis for missing value imputation for coevolving EEG data. J Zhejiang Univ Sci 12(8):687–697CrossRef
go back to reference Kühnert C, Gonuguntla NM, Krieg H, Nowak D, Thomas JA (2021) Application of LSTM networks for water demand prediction in optimal pump control. Water 13(5):644CrossRef Kühnert C, Gonuguntla NM, Krieg H, Nowak D, Thomas JA (2021) Application of LSTM networks for water demand prediction in optimal pump control. Water 13(5):644CrossRef
go back to reference Kumaran SR, Othman MS, Yusuf LM, Yunianta A (2019) Estimation of Missing Values Using Hybrid Fuzzy Clustering Mean and Majority Vote for Microarray Data. Procedia Comput Sci 163:145–153CrossRef Kumaran SR, Othman MS, Yusuf LM, Yunianta A (2019) Estimation of Missing Values Using Hybrid Fuzzy Clustering Mean and Majority Vote for Microarray Data. Procedia Comput Sci 163:145–153CrossRef
go back to reference Luengo J, Sáez JA, Herrera F (2012) Missing data imputation for fuzzy rule-based classification systems. Soft Comput 16(5):863–881CrossRef Luengo J, Sáez JA, Herrera F (2012) Missing data imputation for fuzzy rule-based classification systems. Soft Comput 16(5):863–881CrossRef
go back to reference Madani K (2014) Water management in Iran: what is causing the looming crisis? J Environ Stud Sci 4(4):315–328CrossRef Madani K (2014) Water management in Iran: what is causing the looming crisis? J Environ Stud Sci 4(4):315–328CrossRef
go back to reference Mousavi-Mirkalaei P, Roozbahani A, Banihabib ME, Randhir TO (2022) Forecasting urban water consumption using bayesian networks and gene expression programming. Earth Sci Inf 15(1):623–633CrossRef Mousavi-Mirkalaei P, Roozbahani A, Banihabib ME, Randhir TO (2022) Forecasting urban water consumption using bayesian networks and gene expression programming. Earth Sci Inf 15(1):623–633CrossRef
go back to reference Nejadrekabi M, Eslamian S, Zareian MJ (2022) Spatial statistics techniques for SPEI and NDVI drought indices: a case study of Khuzestan Province. Int J Environ Sci Technol 19(7):6573–6594CrossRef Nejadrekabi M, Eslamian S, Zareian MJ (2022) Spatial statistics techniques for SPEI and NDVI drought indices: a case study of Khuzestan Province. Int J Environ Sci Technol 19(7):6573–6594CrossRef
go back to reference Niknam A, Zare HK, Hosseininasab H, Mostafaeipour A, Herrera M (2022) A Critical Review of Short-Term Water Demand Forecasting Tools—What Method Should I Use? Sustainability 14(9):5412CrossRef Niknam A, Zare HK, Hosseininasab H, Mostafaeipour A, Herrera M (2022) A Critical Review of Short-Term Water Demand Forecasting Tools—What Method Should I Use? Sustainability 14(9):5412CrossRef
go back to reference Pesantez JE, Berglund EZ, Kaza N (2020) Smart meters data for modeling and forecasting water demand at the user-level. Environ Model Softw 125:104633CrossRef Pesantez JE, Berglund EZ, Kaza N (2020) Smart meters data for modeling and forecasting water demand at the user-level. Environ Model Softw 125:104633CrossRef
go back to reference Qi C, Chang NB (2011) System dynamics modeling for municipal water demand estimation in an urban region under uncertain economic impacts. J Environ Manage 92(6):1628–1641CrossRef Qi C, Chang NB (2011) System dynamics modeling for municipal water demand estimation in an urban region under uncertain economic impacts. J Environ Manage 92(6):1628–1641CrossRef
go back to reference Razavi-Far R, Saif M (2016) Imputation of missing data using fuzzy neighborhood density-based clustering. In 2016 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE) (pp. 1834–1841). IEEE Razavi-Far R, Saif M (2016) Imputation of missing data using fuzzy neighborhood density-based clustering. In 2016 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE) (pp. 1834–1841). IEEE
go back to reference Rezaali M, Quilty J, Karimi A (2021) Probabilistic urban water demand forecasting using wavelet-based machine learning models. J Hydrol 600:126358CrossRef Rezaali M, Quilty J, Karimi A (2021) Probabilistic urban water demand forecasting using wavelet-based machine learning models. J Hydrol 600:126358CrossRef
go back to reference Saemian P, Tourian MJ, AghaKouchak A, Madani K, Sneeuw N (2022) How much water did Iran lose over the last two decades? J. Hydrol Reg Stud 41:101095CrossRef Saemian P, Tourian MJ, AghaKouchak A, Madani K, Sneeuw N (2022) How much water did Iran lose over the last two decades? J. Hydrol Reg Stud 41:101095CrossRef
go back to reference Tang F, Ishwaran H (2017) Random forest missing data algorithms. Stat Anal Data Min ASA Data Sci J 10(6):363–377CrossRef Tang F, Ishwaran H (2017) Random forest missing data algorithms. Stat Anal Data Min ASA Data Sci J 10(6):363–377CrossRef
go back to reference Torres JF, Martínez-Álvarez F, Troncoso A (2022) A deep LSTM network for the Spanish electricity consumption forecasting. Neural Comput Appl 34(13):10533–10545CrossRef Torres JF, Martínez-Álvarez F, Troncoso A (2022) A deep LSTM network for the Spanish electricity consumption forecasting. Neural Comput Appl 34(13):10533–10545CrossRef
go back to reference Vijai P, Sivakumar PB (2018) Performance comparison of techniques for water demand forecasting. Procedia Comput Sci 143:258–266CrossRef Vijai P, Sivakumar PB (2018) Performance comparison of techniques for water demand forecasting. Procedia Comput Sci 143:258–266CrossRef
go back to reference Vysala A, Gomes D (2020). Evaluating and validating cluster results. arXiv preprint arXiv:2007.08034 Vysala A, Gomes D (2020). Evaluating and validating cluster results. arXiv preprint arXiv:2007.08034
go back to reference Wang X, Xu Y (2019) An improved index for clustering validation based on Silhouette index and Calinski-Harabasz index. In IOP Conference Series: Materials Science and Engineering (Vol. 569, No. 5, p. 052024). IOP Publishing Wang X, Xu Y (2019) An improved index for clustering validation based on Silhouette index and Calinski-Harabasz index. In IOP Conference Series: Materials Science and Engineering (Vol. 569, No. 5, p. 052024). IOP Publishing
go back to reference Zadeh LA, Klir GJ, Yuan B (1996) Fuzzy sets, fuzzy logic, and fuzzy systems: selected papers (Vol. 6). World scientific Zadeh LA, Klir GJ, Yuan B (1996) Fuzzy sets, fuzzy logic, and fuzzy systems: selected papers (Vol. 6). World scientific
go back to reference Zaidi AZ, Rasmani KA (2016) Classification of excessive domestic water consumption using Fuzzy Clustering Method. In Journal of Physics: Conference Series (Vol. 738, No. 1, p. 012081). IOP Publishing Zaidi AZ, Rasmani KA (2016) Classification of excessive domestic water consumption using Fuzzy Clustering Method. In Journal of Physics: Conference Series (Vol. 738, No. 1, p. 012081). IOP Publishing
go back to reference Zanfei A, Brentan BM, Menapace A, Righetti M, Herrera M (2022a) Graph convolutional recurrent neural networks for water demand forecasting. Water Resour Res 58(7):e2022WRO32299CrossRef Zanfei A, Brentan BM, Menapace A, Righetti M, Herrera M (2022a) Graph convolutional recurrent neural networks for water demand forecasting. Water Resour Res 58(7):e2022WRO32299CrossRef
go back to reference Zanfei A, Menapace A, Brentan BM, Righetti M (2022b) How does missing data imputation affect the forecasting of urban water demand? J Water Resour Plan Manag 148(11):04022060CrossRef Zanfei A, Menapace A, Brentan BM, Righetti M (2022b) How does missing data imputation affect the forecasting of urban water demand? J Water Resour Plan Manag 148(11):04022060CrossRef
go back to reference Zubaidi SL, Ortega-Martorell S, Al-Bugharbee H, Olier I, Hashim KS, Gharghan SK, Al-Khaddar R (2020) Urban water demand prediction for a city that suffers from climate change and population growth: Gauteng province case study. Water, 12(7), 1885 Zubaidi SL, Ortega-Martorell S, Al-Bugharbee H, Olier I, Hashim KS, Gharghan SK, Al-Khaddar R (2020) Urban water demand prediction for a city that suffers from climate change and population growth: Gauteng province case study. Water, 12(7), 1885
go back to reference Yu Y, Si X, Hu C, Zhang J (2019) A review of recurrent neural networks: LSTM cells and network architectures. Neural Comput 31(7):1235–1270CrossRef Yu Y, Si X, Hu C, Zhang J (2019) A review of recurrent neural networks: LSTM cells and network architectures. Neural Comput 31(7):1235–1270CrossRef
Metadata
Title
A hybrid approach combining the multi-dimensional time series k-means algorithm and long short-term memory networks to predict the monthly water demand according to the uncertainty in the dataset
Authors
Azar Niknam
Hasan Khademi Zare
Hassan Hosseininasab
Ali Mostafaeipour
Publication date
10-03-2023
Publisher
Springer Berlin Heidelberg
Published in
Earth Science Informatics / Issue 2/2023
Print ISSN: 1865-0473
Electronic ISSN: 1865-0481
DOI
https://doi.org/10.1007/s12145-023-00976-y

Other articles of this Issue 2/2023

Earth Science Informatics 2/2023 Go to the issue

Premium Partner