Skip to main content
Erschienen in: The Journal of Supercomputing 11/2021

14.04.2021

Sliding window-based LightGBM model for electric load forecasting using anomaly repair

verfasst von: Sungwoo Park, Seungmin Jung, Seungwon Jung, Seungmin Rho, Eenjun Hwang

Erschienen in: The Journal of Supercomputing | Ausgabe 11/2021

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Smart grids have attracted much attention recently for their potential to reduce power system operating and management costs. Smart grid core components include energy storage, renewable energy source(s), and smart meters. Smart meters collect diverse data regarding smart grid operation, which can lead to inefficient operation if the meter data are damaged or tampered with during collection or transmission. Therefore, it is important to identify abnormalities in smart grid data and process them accordingly. Various anomaly detection models have been proposed using statistical methods, but they cannot detect some anomaly patterns accurately, and the models generally did not consider repair strategies for the detected anomalies. Anomaly repair should be included with model training to improve forecasting performance. This paper proposes a robust sliding window-based LightGBM model for short-term load forecasting using anomaly detection and repair. We first show how to detect anomalies using a variational autoencoder and then how they can be repaired using a random forest method. Finally, we verify that the proposed sliding window-based LightGBM achieves superior forecasting performance in combination with anomaly detection and repair.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Hodge VJ, O’Keefe S, Weeks M, Moulds A (2014) Wireless sensor networks for condition monitoring in the railway industry: a survey. IEEE Trans Intell Transp Syst 16(3):1088–1106CrossRef Hodge VJ, O’Keefe S, Weeks M, Moulds A (2014) Wireless sensor networks for condition monitoring in the railway industry: a survey. IEEE Trans Intell Transp Syst 16(3):1088–1106CrossRef
2.
Zurück zum Zitat Hempstead M, Lyons MJ, Brooks D, Wei GY (2008) Survey of hardware systems for wireless sensor networks. J Low Power Electron 4(1):11–20CrossRef Hempstead M, Lyons MJ, Brooks D, Wei GY (2008) Survey of hardware systems for wireless sensor networks. J Low Power Electron 4(1):11–20CrossRef
3.
Zurück zum Zitat Jagannathan S (2016) Real-time big data analytics architecture for remote sensing application. In: International Conference on Signal Processing, Communication, Power and Embedded System, pp 1912–1916 (2016). Jagannathan S (2016) Real-time big data analytics architecture for remote sensing application. In: International Conference on Signal Processing, Communication, Power and Embedded System, pp 1912–1916 (2016).
4.
Zurück zum Zitat Kanoun O, Trankler HR (2004) Sensor technology advances and future trends. IEEE Trans Instrum Meas 53(6):1497–1501CrossRef Kanoun O, Trankler HR (2004) Sensor technology advances and future trends. IEEE Trans Instrum Meas 53(6):1497–1501CrossRef
5.
Zurück zum Zitat Bell WR, Hillmer SC (1983) Modeling time series with calendar variation. J Am Stat Assoc 78(383):526–534CrossRef Bell WR, Hillmer SC (1983) Modeling time series with calendar variation. J Am Stat Assoc 78(383):526–534CrossRef
6.
Zurück zum Zitat Fu TC (2011) A review on time series data mining. Eng Appl Artif Intell 24(1):164–181CrossRef Fu TC (2011) A review on time series data mining. Eng Appl Artif Intell 24(1):164–181CrossRef
7.
Zurück zum Zitat Ding D, Cooper RA, Pasquina PF, Fici-Pasquina L (2011) Sensor technology for smart homes. Maturitas 69(2):131–136CrossRef Ding D, Cooper RA, Pasquina PF, Fici-Pasquina L (2011) Sensor technology for smart homes. Maturitas 69(2):131–136CrossRef
8.
Zurück zum Zitat Yu Z, Zheng X, Huang F, Guo W, Sun L, Yu Z (2020) A framework based on sparse representation model for time series prediction in smart city. Front Comp Sci 15(1):1–13 Yu Z, Zheng X, Huang F, Guo W, Sun L, Yu Z (2020) A framework based on sparse representation model for time series prediction in smart city. Front Comp Sci 15(1):1–13
9.
Zurück zum Zitat Chou JS, Ngo NT (2016) Smart grid data analytics framework for increasing energy savings in residential buildings. Autom Constr 72:247–257CrossRef Chou JS, Ngo NT (2016) Smart grid data analytics framework for increasing energy savings in residential buildings. Autom Constr 72:247–257CrossRef
10.
Zurück zum Zitat Tabrizchi H, Javidi MM, Amirzadeh V (2019) Estimates of residential building energy consumption using a multi-verse optimizer-based support vector machine with k-fold cross-validation. Evolv Syst 1–13 (2019) Tabrizchi H, Javidi MM, Amirzadeh V (2019) Estimates of residential building energy consumption using a multi-verse optimizer-based support vector machine with k-fold cross-validation. Evolv Syst 1–13 (2019)
11.
Zurück zum Zitat Park S, Moon J, Hwang E (2019) 2-Stage electric load forecasting scheme for day-ahead CCHP scheduling. In: IEEE 13th International Conference on Power Electronics and Drive Systems (PEDS), pp 1–4 Park S, Moon J, Hwang E (2019) 2-Stage electric load forecasting scheme for day-ahead CCHP scheduling. In: IEEE 13th International Conference on Power Electronics and Drive Systems (PEDS), pp 1–4
12.
Zurück zum Zitat Montazerolghaem A, Moghaddam MHY, Leon-Garcia A (2017) OpenAMI: Software-defined AMI load balancing. IEEE Internet Things J 5(1):206–218CrossRef Montazerolghaem A, Moghaddam MHY, Leon-Garcia A (2017) OpenAMI: Software-defined AMI load balancing. IEEE Internet Things J 5(1):206–218CrossRef
13.
Zurück zum Zitat Raciti M, Nadjm-Tehrani S (2013) Embedded cyber-physical anomaly detection in smart meters. In: Critical information infrastructures security, pp 34–45 Raciti M, Nadjm-Tehrani S (2013) Embedded cyber-physical anomaly detection in smart meters. In: Critical information infrastructures security, pp 34–45
14.
Zurück zum Zitat Jiang R, Lu R, Wang Y, Luo J, Shen C, Shen X (2014) Energy-theft detection issues for advanced metering infrastructure in smart grid. Tsinghua Sci Technol 19(2):105–120CrossRef Jiang R, Lu R, Wang Y, Luo J, Shen C, Shen X (2014) Energy-theft detection issues for advanced metering infrastructure in smart grid. Tsinghua Sci Technol 19(2):105–120CrossRef
15.
Zurück zum Zitat Moghaddass R, Wang J (2017) A hierarchical framework for smart grid anomaly detection using large-scale smart meter data. IEEE Trans Smart Grid 9(6):5820–5830CrossRef Moghaddass R, Wang J (2017) A hierarchical framework for smart grid anomaly detection using large-scale smart meter data. IEEE Trans Smart Grid 9(6):5820–5830CrossRef
16.
Zurück zum Zitat Zhang W, Yang Q, Geng Y (2009) A survey of anomaly detection methods in networks. In: International symposium on computer network and multimedia technology, pp 1–3 Zhang W, Yang Q, Geng Y (2009) A survey of anomaly detection methods in networks. In: International symposium on computer network and multimedia technology, pp 1–3
17.
Zurück zum Zitat Wang C, Viswanathan K, Choudur L, Talwar V, Satterfield W, Schwan K (2011) Statistical techniques for online anomaly detection in data centers. In: 12th IFIP/IEEE international symposium on integrated network management and workshops, pp 385–392 Wang C, Viswanathan K, Choudur L, Talwar V, Satterfield W, Schwan K (2011) Statistical techniques for online anomaly detection in data centers. In: 12th IFIP/IEEE international symposium on integrated network management and workshops, pp 385–392
18.
Zurück zum Zitat Chandola V, Banerjee A, Kumar V (2009) Anomaly detection: a survey. ACM Comput Surv (CSUR) 41(3):1–58CrossRef Chandola V, Banerjee A, Kumar V (2009) Anomaly detection: a survey. ACM Comput Surv (CSUR) 41(3):1–58CrossRef
19.
Zurück zum Zitat Zhang A, Song S, Wang J, Yu PS (2017) Time series data cleaning: from anomaly detection to anomaly repairing. Proc VLDB Endowm 10(10):1046–1057CrossRef Zhang A, Song S, Wang J, Yu PS (2017) Time series data cleaning: from anomaly detection to anomaly repairing. Proc VLDB Endowm 10(10):1046–1057CrossRef
20.
Zurück zum Zitat Jung S, Moon J, Park S, Rho S, Baik SW, Hwang E (2020) Bagging ensemble of multilayer perceptrons for missing electricity consumption data imputation. Sensors 20(6):1772CrossRef Jung S, Moon J, Park S, Rho S, Baik SW, Hwang E (2020) Bagging ensemble of multilayer perceptrons for missing electricity consumption data imputation. Sensors 20(6):1772CrossRef
21.
Zurück zum Zitat Armstrong JS (1989) Combining forecasts: the end of the beginning or the beginning of the end? Int J Forecast 5:585–588CrossRef Armstrong JS (1989) Combining forecasts: the end of the beginning or the beginning of the end? Int J Forecast 5:585–588CrossRef
22.
Zurück zum Zitat Breunig MM, Kriegel HP, Ng RT, Sander J (2000) LOF: identifying density-based local outliers. In: Proceedings of the 2000 ACM SIGMOD International Conference on Management of Data, pp 93–104 Breunig MM, Kriegel HP, Ng RT, Sander J (2000) LOF: identifying density-based local outliers. In: Proceedings of the 2000 ACM SIGMOD International Conference on Management of Data, pp 93–104
23.
Zurück zum Zitat Liu FT, Ting KM, Zhou ZH (2008) Isolation forest. In: 8th IEEE International Conference on Data Mining, pp 413–422 Liu FT, Ting KM, Zhou ZH (2008) Isolation forest. In: 8th IEEE International Conference on Data Mining, pp 413–422
24.
Zurück zum Zitat Chen J, Sathe S, Aggarwal C, Turaga D (2017) Outlier detection with autoencoder ensembles. In: Proceedings of the SIAM International Conference on Data Mining, pp 90–98 Chen J, Sathe S, Aggarwal C, Turaga D (2017) Outlier detection with autoencoder ensembles. In: Proceedings of the SIAM International Conference on Data Mining, pp 90–98
25.
Zurück zum Zitat Akouemo HN, Povinelli RJ (2017) Data improving in time series using ARX and ANN models. IEEE Trans Power Syst 32(5):3352–3359CrossRef Akouemo HN, Povinelli RJ (2017) Data improving in time series using ARX and ANN models. IEEE Trans Power Syst 32(5):3352–3359CrossRef
26.
Zurück zum Zitat Araya DB, Grolinger K, ElYamany HF, Capretz MA, Bitsuamlak G (2017) An ensemble learning framework for anomaly detection in building energy consumption. Energy Build 144:191–206CrossRef Araya DB, Grolinger K, ElYamany HF, Capretz MA, Bitsuamlak G (2017) An ensemble learning framework for anomaly detection in building energy consumption. Energy Build 144:191–206CrossRef
27.
Zurück zum Zitat Xu CD, Wang JF, Hu MG, Li QX (2013) Interpolation of missing temperature data at meteorological stations using P-BSHADE. J Clim 26(19):7452–7463CrossRef Xu CD, Wang JF, Hu MG, Li QX (2013) Interpolation of missing temperature data at meteorological stations using P-BSHADE. J Clim 26(19):7452–7463CrossRef
28.
Zurück zum Zitat Habermann C, Kindermann F (2007) Multidimensional spline interpolation: theory and applications. Comput Econ 30(2):153–169CrossRef Habermann C, Kindermann F (2007) Multidimensional spline interpolation: theory and applications. Comput Econ 30(2):153–169CrossRef
29.
Zurück zum Zitat Gan S, Wang S, Chen Y, Zhang Y, Jin Z (2015) Dealiased seismic data interpolation using seislet transform with low-frequency constraint. IEEE Geosci Remote Sens Lett 12(10):2150–2154CrossRef Gan S, Wang S, Chen Y, Zhang Y, Jin Z (2015) Dealiased seismic data interpolation using seislet transform with low-frequency constraint. IEEE Geosci Remote Sens Lett 12(10):2150–2154CrossRef
30.
Zurück zum Zitat Jurado S, Peralta J, Nebot A, Mugica F, Cortez P (2013) Short-term electric load forecasting using computational intelligence methods. In: IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), pp 1–8 Jurado S, Peralta J, Nebot A, Mugica F, Cortez P (2013) Short-term electric load forecasting using computational intelligence methods. In: IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), pp 1–8
31.
Zurück zum Zitat Grolinger K, L’Heureux A, Capretz MA, Seewald L (2016) Energy forecasting for event venues: Big data and prediction accuracy. Energy Build 112:222–233CrossRef Grolinger K, L’Heureux A, Capretz MA, Seewald L (2016) Energy forecasting for event venues: Big data and prediction accuracy. Energy Build 112:222–233CrossRef
32.
Zurück zum Zitat Abbasi RA, Javaid N, Ghuman MNJ, Khan ZA, Rehman SU (2019) Short term load forecasting using XGBoost. In: Workshops of the International Conference on Advanced Information Networking and Applications, pp 1120–1131 Abbasi RA, Javaid N, Ghuman MNJ, Khan ZA, Rehman SU (2019) Short term load forecasting using XGBoost. In: Workshops of the International Conference on Advanced Information Networking and Applications, pp 1120–1131
33.
Zurück zum Zitat Kuo PH, Huang CJ (2018) A high precision artificial neural networks model for short-term energy load forecasting. Energies 11(1):213CrossRef Kuo PH, Huang CJ (2018) A high precision artificial neural networks model for short-term energy load forecasting. Energies 11(1):213CrossRef
34.
Zurück zum Zitat Massana J, Pous C, Burgas L, Melendez J, Colomer J (2015) Short-term load forecasting in a non-residential building contrasting models and attributes. Energy Build 92:322–330CrossRef Massana J, Pous C, Burgas L, Melendez J, Colomer J (2015) Short-term load forecasting in a non-residential building contrasting models and attributes. Energy Build 92:322–330CrossRef
35.
Zurück zum Zitat Park S, Moon J, Jung S, Rho S, Baik SW, Hwang E (2020) A two-stage industrial load forecasting scheme for day-ahead combined cooling, heating and power scheduling. Energies 13(2):443CrossRef Park S, Moon J, Jung S, Rho S, Baik SW, Hwang E (2020) A two-stage industrial load forecasting scheme for day-ahead combined cooling, heating and power scheduling. Energies 13(2):443CrossRef
36.
Zurück zum Zitat Wang P, Liu B, Hong T (2016) Electric load forecasting with recency effect: a big data approach. Int J Forecast 32:585–597CrossRef Wang P, Liu B, Hong T (2016) Electric load forecasting with recency effect: a big data approach. Int J Forecast 32:585–597CrossRef
37.
Zurück zum Zitat Xie J, Chen Y, Hong T, Laing TD (2016) Relative humidity for load forecasting models. IEEE Trans Smart Grid 9:191–198CrossRef Xie J, Chen Y, Hong T, Laing TD (2016) Relative humidity for load forecasting models. IEEE Trans Smart Grid 9:191–198CrossRef
38.
Zurück zum Zitat Kramer MA (1991) Nonlinear principal component analysis using autoassociative neural networks. AIChE J 37(2):233–243CrossRef Kramer MA (1991) Nonlinear principal component analysis using autoassociative neural networks. AIChE J 37(2):233–243CrossRef
40.
Zurück zum Zitat Ke G, Meng Q, Finley T, Wang T, Chen W, Ma W, Ye Q, Liu TY (2017) LightGBM: a highly efficient gradient boosting decision tree. In: Advances in neural information processing systems, pp 3146–3154 Ke G, Meng Q, Finley T, Wang T, Chen W, Ma W, Ye Q, Liu TY (2017) LightGBM: a highly efficient gradient boosting decision tree. In: Advances in neural information processing systems, pp 3146–3154
41.
Zurück zum Zitat Moon J, Jung S, Rew J, Rho S, Hwang E (2020) Combination of short term load forecasting models based on a stacking ensemble approach. Energy Build 109921 Moon J, Jung S, Rew J, Rho S, Hwang E (2020) Combination of short term load forecasting models based on a stacking ensemble approach. Energy Build 109921
Metadaten
Titel
Sliding window-based LightGBM model for electric load forecasting using anomaly repair
verfasst von
Sungwoo Park
Seungmin Jung
Seungwon Jung
Seungmin Rho
Eenjun Hwang
Publikationsdatum
14.04.2021
Verlag
Springer US
Erschienen in
The Journal of Supercomputing / Ausgabe 11/2021
Print ISSN: 0920-8542
Elektronische ISSN: 1573-0484
DOI
https://doi.org/10.1007/s11227-021-03787-4

Weitere Artikel der Ausgabe 11/2021

The Journal of Supercomputing 11/2021 Zur Ausgabe

Premium Partner