Skip to main content

2021 | OriginalPaper | Buchkapitel

Imputation of Rainfall Data Using Improved Neural Network Algorithm

verfasst von : Po Chan Chiu, Ali Selamat, Ondrej Krejcar, King Kuok Kuok

Erschienen in: Pattern Recognition. ICPR International Workshops and Challenges

Verlag: Springer International Publishing

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Missing rainfall data have reduced the quality of hydrological data analysis because they are the essential input for hydrological modeling. Much research has focused on rainfall data imputation. However, the compatibility of precipitation (rainfall) and non-precipitation (meteorology) as input data has received less attention. First, we propose a novel input structure for the missing data imputation method. Principal component analysis (PCA) is used to extract the most relevant features from the meteorological data. This paper introduces the combined input of the significant principal components (PCs) and rainfall data from nearest neighbor gauging stations as the input to the estimation of the missing values. Second, the effects of the combination input for infilling the missing rainfall data series were compared using the sine cosine algorithm neural network (SCANN) and feedforward neural network (FFNN). The results showed that SCANN outperformed FFNN imputation in terms of mean absolute error (MAE), root means square error (RMSE) and correlation coefficient (R), with an average accuracy of more than 90%. This study revealed that as the percentage of missingness increased, the precision of both imputation methods reduced.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Muñoz, P., Orellana-Alvear, J., Willems, P., Célleri, R.: Flash-flood forecasting in an Andean mountain catchment—development of a step-wise methodology based on the random forest algorithm. Water 10(11), 1519 (2018)CrossRef Muñoz, P., Orellana-Alvear, J., Willems, P., Célleri, R.: Flash-flood forecasting in an Andean mountain catchment—development of a step-wise methodology based on the random forest algorithm. Water 10(11), 1519 (2018)CrossRef
2.
Zurück zum Zitat Szewrański, S., Chruściński, J., Kazak, J., Świąder, M., Tokarczyk-Dorociak, K., Żmuda, R.: Pluvial Flood Risk Assessment Tool (PFRA) for rainwater management and adaptation to climate change in newly urbanised areas. Water 10(4), 386 (2018)CrossRef Szewrański, S., Chruściński, J., Kazak, J., Świąder, M., Tokarczyk-Dorociak, K., Żmuda, R.: Pluvial Flood Risk Assessment Tool (PFRA) for rainwater management and adaptation to climate change in newly urbanised areas. Water 10(4), 386 (2018)CrossRef
3.
Zurück zum Zitat Kuok, K.K.: Parameter Optimization Methods for Calibrating Tank Model and Neural Network Model for Rainfall-runoff Modeling. Doctoral dissertation, Ph.D. thesis. Universiti Technology Malaysia (2010) Kuok, K.K.: Parameter Optimization Methods for Calibrating Tank Model and Neural Network Model for Rainfall-runoff Modeling. Doctoral dissertation, Ph.D. thesis. Universiti Technology Malaysia (2010)
4.
Zurück zum Zitat Mcdonald, R.A., Thurston, P.W., Nelson, M.R.A.: Monte Carlo study of missing item methods. Organizational Res. Methods 3(1), 71–92 (2000) Mcdonald, R.A., Thurston, P.W., Nelson, M.R.A.: Monte Carlo study of missing item methods. Organizational Res. Methods 3(1), 71–92 (2000)
5.
Zurück zum Zitat McKnight, P.E., McKnight, K.M., Sidani, S., Figueredo, A.J.: Missing Data: A Gentle Introduction. Guilford Press (2007). McKnight, P.E., McKnight, K.M., Sidani, S., Figueredo, A.J.: Missing Data: A Gentle Introduction. Guilford Press (2007).
6.
Zurück zum Zitat Lee, K.J., Carlin, J.B.: Multiple imputation for missing data: fully conditional specification versus multivariate normal imputation. Am. J. Epidemiol. 171(5), 624–632 (2010)CrossRef Lee, K.J., Carlin, J.B.: Multiple imputation for missing data: fully conditional specification versus multivariate normal imputation. Am. J. Epidemiol. 171(5), 624–632 (2010)CrossRef
8.
Zurück zum Zitat Mispan, M.R., Rahman, N.F.A., Ali, M.F., Khalid, K., Bakar, M.H.A., Haron, S.H.: Missing river discharge data imputation approach using artificial neural network. Methodology 25, 20 (2015) Mispan, M.R., Rahman, N.F.A., Ali, M.F., Khalid, K., Bakar, M.H.A., Haron, S.H.: Missing river discharge data imputation approach using artificial neural network. Methodology 25, 20 (2015)
9.
Zurück zum Zitat Chiu, P.C., Selamat, A., Krejcar, O.: Infilling missing rainfall and runoff data for sarawak, malaysia using gaussian mixture model based k-nearest neighbor imputation. In: Wotawa, F., Friedrich, G., Pill, I., Koitz-Hristov, R., Ali, M. (eds.) IEA/AIE 2019. LNCS (LNAI), vol. 11606, pp. 27–38. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-22999-3_3CrossRef Chiu, P.C., Selamat, A., Krejcar, O.: Infilling missing rainfall and runoff data for sarawak, malaysia using gaussian mixture model based k-nearest neighbor imputation. In: Wotawa, F., Friedrich, G., Pill, I., Koitz-Hristov, R., Ali, M. (eds.) IEA/AIE 2019. LNCS (LNAI), vol. 11606, pp. 27–38. Springer, Cham (2019). https://​doi.​org/​10.​1007/​978-3-030-22999-3_​3CrossRef
11.
Zurück zum Zitat Mirjalili, S.: SCA: a sine cosine algorithm for solving optimization problems. Knowl.-Based Syst. 96, 120–133 (2016)CrossRef Mirjalili, S.: SCA: a sine cosine algorithm for solving optimization problems. Knowl.-Based Syst. 96, 120–133 (2016)CrossRef
12.
Zurück zum Zitat Qu, C., Zeng, Z., Dai, J., Yi, Z., He, W.: A modified sine-cosine algorithm based on neighborhood search and greedy levy mutation. Computational intelligence and neuroscience (2018) Qu, C., Zeng, Z., Dai, J., Yi, Z., He, W.: A modified sine-cosine algorithm based on neighborhood search and greedy levy mutation. Computational intelligence and neuroscience (2018)
13.
Zurück zum Zitat Das, S., Bhattacharya, A., Chakraborty, A.K.: Solution of short-term hydrothermal scheduling using sine cosine algorithm. Soft Comput. 22(19), 6409–6427 (2018) Das, S., Bhattacharya, A., Chakraborty, A.K.: Solution of short-term hydrothermal scheduling using sine cosine algorithm. Soft Comput. 22(19), 6409–6427 (2018)
14.
Zurück zum Zitat Li, S., Fang, H., Liu, X.: Parameter optimization of support vector regression based on sine cosine algorithm. Expert Syst. Appl. 91, 63–77 (2018)CrossRef Li, S., Fang, H., Liu, X.: Parameter optimization of support vector regression based on sine cosine algorithm. Expert Syst. Appl. 91, 63–77 (2018)CrossRef
16.
Zurück zum Zitat Chandler, R.E., Isham, V.S., Leith, N.A., Northrop, P.J., Onof, C.J., Wheater, H.S.: Uncertainty in Rainfall Inputs. World Scientific/Imperial College Press, London (2011) Chandler, R.E., Isham, V.S., Leith, N.A., Northrop, P.J., Onof, C.J., Wheater, H.S.: Uncertainty in Rainfall Inputs. World Scientific/Imperial College Press, London (2011)
18.
Zurück zum Zitat Kashiwao, T., Nakayama, K., Ando, S., Ikeda, K., Lee, M., Bahadori, A.: A neural network-based local rainfall prediction system using meteorological data on the Internet: a case study using data from the Japan Meteorological Agency. Appl. Soft Comput. 56, 317–330 (2017)CrossRef Kashiwao, T., Nakayama, K., Ando, S., Ikeda, K., Lee, M., Bahadori, A.: A neural network-based local rainfall prediction system using meteorological data on the Internet: a case study using data from the Japan Meteorological Agency. Appl. Soft Comput. 56, 317–330 (2017)CrossRef
19.
Zurück zum Zitat Yen, M.H., Liu, D.W., Hsin, Y.C., Lin, C.E., Chen, C.C.: Application of the deep learning for the prediction of rainfall in Southern Taiwan. Sci. Rep. 9(1), 1–9 (2019)CrossRef Yen, M.H., Liu, D.W., Hsin, Y.C., Lin, C.E., Chen, C.C.: Application of the deep learning for the prediction of rainfall in Southern Taiwan. Sci. Rep. 9(1), 1–9 (2019)CrossRef
20.
Zurück zum Zitat Grange, S.K., Carslaw, D.C.: Using meteorological normalisation to detect interventions in air quality time series. Sci. Total Environ. 653, 578–588 (2019)CrossRef Grange, S.K., Carslaw, D.C.: Using meteorological normalisation to detect interventions in air quality time series. Sci. Total Environ. 653, 578–588 (2019)CrossRef
21.
Zurück zum Zitat Londhe, S., Dixit, P., Shah, S., Narkhede, S.: Infilling of missing daily rainfall records using artificial neural network. ISH J. Hydraulic Eng. 21(3), 255–264 (2015) Londhe, S., Dixit, P., Shah, S., Narkhede, S.: Infilling of missing daily rainfall records using artificial neural network. ISH J. Hydraulic Eng. 21(3), 255–264 (2015)
22.
Zurück zum Zitat Canchala-Nastar, T., Carvajal-Escobar, Y., Alfonso-Morales, W., Cerón, W.L., Caicedo, E.: Estimation of missing data of monthly rainfall in southwestern Colombia using artificial neural networks. Data Brief 26, 104517 (2019)CrossRef Canchala-Nastar, T., Carvajal-Escobar, Y., Alfonso-Morales, W., Cerón, W.L., Caicedo, E.: Estimation of missing data of monthly rainfall in southwestern Colombia using artificial neural networks. Data Brief 26, 104517 (2019)CrossRef
23.
Zurück zum Zitat Chiu, P.C., Selamat, A., Krejcar, O., Kuok, K.K.: Missing rainfall data estimation using artificial neural network and nearest neighbor imputation. In: Advancing Technology Industrialization Through Intelligent Software Methodologies, Tools and Techniques: Proceedings of the 18th International Conference on New Trends in Intelligent Software Methodologies, Tools and Techniques (SoMeT_19), 318, 132. IOS Press (2019) Chiu, P.C., Selamat, A., Krejcar, O., Kuok, K.K.: Missing rainfall data estimation using artificial neural network and nearest neighbor imputation. In: Advancing Technology Industrialization Through Intelligent Software Methodologies, Tools and Techniques: Proceedings of the 18th International Conference on New Trends in Intelligent Software Methodologies, Tools and Techniques (SoMeT_19), 318, 132. IOS Press (2019)
24.
Zurück zum Zitat Henry, A.J., Hevelone, N.D., Lipsitz, S., Nguyen, L.L.: Comparative methods for handling missing data in large databases. J. Vasc. Surg. 58(5), 1353–1359 (2013)CrossRef Henry, A.J., Hevelone, N.D., Lipsitz, S., Nguyen, L.L.: Comparative methods for handling missing data in large databases. J. Vasc. Surg. 58(5), 1353–1359 (2013)CrossRef
25.
Zurück zum Zitat Cheema, J.R.: Some general guidelines for choosing missing data handling methods in educational research. J. Mod. Appl. Stat. Meth. 13(2), 3 (2014)CrossRef Cheema, J.R.: Some general guidelines for choosing missing data handling methods in educational research. J. Mod. Appl. Stat. Meth. 13(2), 3 (2014)CrossRef
26.
Zurück zum Zitat Zhu, P., Xu, Q., Hu, Q., Zhang, C., Zhao, H.: Multi-label feature selection with missing labels. Pattern Recogn. 74, 488–502 (2018)CrossRef Zhu, P., Xu, Q., Hu, Q., Zhang, C., Zhao, H.: Multi-label feature selection with missing labels. Pattern Recogn. 74, 488–502 (2018)CrossRef
27.
Zurück zum Zitat Hassani, H., Kalantari, M., Ghodsi, Z.: Evaluating the performance of multiple imputation methods for handling missing values in time series data: a study focused on East Africa. Soil-Carbonate-Stable Isotope Data. Stats. 2(4), 457–467 (2019) Hassani, H., Kalantari, M., Ghodsi, Z.: Evaluating the performance of multiple imputation methods for handling missing values in time series data: a study focused on East Africa. Soil-Carbonate-Stable Isotope Data. Stats. 2(4), 457–467 (2019)
28.
Zurück zum Zitat Oba, S., Sato, M.A., Takemasa, I., Monden, M., Matsubara, K.I., Ishii, S.: A Bayesian missing value estimation method for gene expression profile data. Bioinformatics 19(16), 2088–2096 (2003)CrossRef Oba, S., Sato, M.A., Takemasa, I., Monden, M., Matsubara, K.I., Ishii, S.: A Bayesian missing value estimation method for gene expression profile data. Bioinformatics 19(16), 2088–2096 (2003)CrossRef
29.
Zurück zum Zitat Little, R.J., Rubin, D.B.: Statistical Analysis with Missing Data. Wiley, New York (2014) Little, R.J., Rubin, D.B.: Statistical Analysis with Missing Data. Wiley, New York (2014)
30.
Zurück zum Zitat Kurita, T.: Principal Component Analysis (PCA). In: Ikeuchi, K. (eds) Computer Vision. Springer, Boston (2014) Kurita, T.: Principal Component Analysis (PCA). In: Ikeuchi, K. (eds) Computer Vision. Springer, Boston (2014)
31.
Zurück zum Zitat Pearson, K.: Principal components analysis. London, Edinburgh, Dublin Philos. Mag. J. Sci. 6(2), 559 (1901)CrossRef Pearson, K.: Principal components analysis. London, Edinburgh, Dublin Philos. Mag. J. Sci. 6(2), 559 (1901)CrossRef
32.
Zurück zum Zitat Hotelling, H.: Analysis of a complex of statistical variables into principal components. J. Educ. Psychol. 24, 417 (1933)CrossRef Hotelling, H.: Analysis of a complex of statistical variables into principal components. J. Educ. Psychol. 24, 417 (1933)CrossRef
34.
Zurück zum Zitat Khattree, R., Naik, D.N.: Multivariate Data Reduction and Discrimination with SAS Software. Cary, N.C., SAS Institute (2000) Khattree, R., Naik, D.N.: Multivariate Data Reduction and Discrimination with SAS Software. Cary, N.C., SAS Institute (2000)
35.
Zurück zum Zitat Jamil, M., Yang, X.S.: A literature survey of benchmark functions for global optimisation problems. Int. J. Math. Modell. Numer. Optim. 4(2), 150–194 (2013)MATH Jamil, M., Yang, X.S.: A literature survey of benchmark functions for global optimisation problems. Int. J. Math. Modell. Numer. Optim. 4(2), 150–194 (2013)MATH
36.
Zurück zum Zitat Zuśka, Z., Kopcińska, J., Dacewicz, E., Skowera, B., Wojkowski, J., Ziernicka–Wojtaszek, A.: Application of the principal component analysis (PCA) method to assess the impact of meteorological elements on concentrations of particulate matter (PM10): a case study of the Mountain Valley (the Sącz Basin, Poland). Sustainability 11, 6740 (2019) Zuśka, Z., Kopcińska, J., Dacewicz, E., Skowera, B., Wojkowski, J., Ziernicka–Wojtaszek, A.: Application of the principal component analysis (PCA) method to assess the impact of meteorological elements on concentrations of particulate matter (PM10): a case study of the Mountain Valley (the Sącz Basin, Poland). Sustainability 11, 6740 (2019)
37.
Zurück zum Zitat De Silva, C.C., Beckman, S.P., Liu, S., Bowler, N.: Principal component analysis (PCA) as a statistical tool for identifying key indicators of nuclear power plant cable insulation degradation. In: Proceedings of the 18th International Conference on Environmental Degradation of Materials in Nuclear Power Systems–Water Reactors, pp. 1227–1239. Springer, Cham (2019) De Silva, C.C., Beckman, S.P., Liu, S., Bowler, N.: Principal component analysis (PCA) as a statistical tool for identifying key indicators of nuclear power plant cable insulation degradation. In: Proceedings of the 18th International Conference on Environmental Degradation of Materials in Nuclear Power Systems–Water Reactors, pp. 1227–1239. Springer, Cham (2019)
38.
Zurück zum Zitat Gill, M.K., Asefa, T., Kaheil, Y., McKee, M.: Effect of missing data on performance of learning algorithms for hydrologic predictions: implications to an imputation technique. Water Resour. Res. 43(7) (2007) Gill, M.K., Asefa, T., Kaheil, Y., McKee, M.: Effect of missing data on performance of learning algorithms for hydrologic predictions: implications to an imputation technique. Water Resour. Res. 43(7) (2007)
39.
Zurück zum Zitat Kim, T., Ko, W., Kim, J.: Analysis and impact evaluation of missing data imputation in day-ahead PV generation forecasting. Appl. Sci. 9(1), 204 (2019)CrossRef Kim, T., Ko, W., Kim, J.: Analysis and impact evaluation of missing data imputation in day-ahead PV generation forecasting. Appl. Sci. 9(1), 204 (2019)CrossRef
40.
Zurück zum Zitat Ayilara, O.F., Zhang, L., Sajobi, T.T., Sawatzky, R., Bohm, E., Lix, L.M.: Impact of missing data on bias and precision when estimating change in patient-reported outcomes from a clinical registry. Health Quality Life Outcomes 17(1), 106 (2019)CrossRef Ayilara, O.F., Zhang, L., Sajobi, T.T., Sawatzky, R., Bohm, E., Lix, L.M.: Impact of missing data on bias and precision when estimating change in patient-reported outcomes from a clinical registry. Health Quality Life Outcomes 17(1), 106 (2019)CrossRef
Metadaten
Titel
Imputation of Rainfall Data Using Improved Neural Network Algorithm
verfasst von
Po Chan Chiu
Ali Selamat
Ondrej Krejcar
King Kuok Kuok
Copyright-Jahr
2021
DOI
https://doi.org/10.1007/978-3-030-68799-1_28