Skip to main content

Advertisement

SpringerLink
Log in

Smart electrical grids based on cloud, IoT, and big data technologies: state of the art

  • Original Research
  • Published:
Journal of Ambient Intelligence and Humanized Computing Aims and scope Submit manuscript

Abstract

The smart electrical grid (SEG), that utilizes information for creating a widely distributed automated energy delivery network, is considered as an advanced digital 2-way power flow power system. Under different uncertainties, SEG is capable of self-healing, adaptive, resilient, and sustainable with foresight for prediction. Hence, SEG is considered as the next generation power grid. In this paper, a comprehensive survey on SEG as a new technology and operating models which will affect performance of distribution networks in the future are explored in detail. Most of the basic concepts affect such new technology like (Internet of Things (IoT), fog, cloud computing, and big data analysis) are discussed. A brief overview of IoT technologies is provided. It will explore the architectural structure of a typical IoT, cloud computing system, and different levels of the system. Furthermore, many classification methods and then electrical load forecasting (ELF) strategy that includes the preprocessing phase and the prediction phase have been discussed. Additionally, the different techniques used to manage big data generated by sensors and meters for application processing are explored. Feature selection and outlier rejection are discussed as a preprocessing process to filter the data, and then the load prediction process is explained. Finally, this paper covers the analysis of the load prediction phase in ELF strategy in which the prediction techniques will be reviewed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20

Similar content being viewed by others

References

  • Akhavan-Hejazi H, Mohsenian-Rad H (2018) Power systems big data analytics: an assessment of paradigm shift barriers and prospects. Energy Rep 4:91–100

    Article  Google Scholar 

  • Al Yami M, Schaefer D (2019) Fog computing as a complementary approach to cloud computing. In: Proceedings of the 2019 International Conference on Computer and Information Sciences (ICCIS).

  • Ali D, Yohanna M, Ijasini P, Garkida M (2018) Application of fuzzy – Neuro to model weather parameter variability impacts on electrical load based on long-term forecasting. Alex Eng J 57(1):223–233

    Article  Google Scholar 

  • Ali D, Yohanna M, Puwu M, Garkida B (2016) Long-term load forecast modelling using a fuzzy logic approach. Pac Sci Rev A Nat Sci Eng 18(2):123–127

    Google Scholar 

  • Alrawais A, Alhothaily A, Hu C, Xing X, Cheng X (2017) An attribute-based encryption scheme to secure fog communications. IEEE Access 5:9131–9138

    Article  Google Scholar 

  • Alyam R, Alhajja J, Alnajran B, Elaalam I, Alqahtan A, Aldhaffer N, Owolab T, Olatun S (2017) Investigating the effect of correlation based feature selection on breast cancer diagnosis using artificial neural network and support vector machines. In: Proceedings of the 2017 International Conference on Informatics, Health & Technology (ICIHT). IEEE, Riyadh, pp 1–7

  • Ansari M, Vakili V, Bahrak B (2019) Evaluation of big data frameworks for analysis of smart grids. J Big Data 6(109):1–14

    Google Scholar 

  • Arnold M, Rui H, Wellssow W (2011) An approach to smart grid metrics. In: Proceedings of the 2011 2nd IEEE PES International Conference and Exhibition on Innovative Smart Grid Technologies, Manchester, pp 1–7

  • Atlam H, Walters R, Wills G (2018) Fog computing and the internet of things: a review. Big Data Cogn Comput 2(10):1–18

    Google Scholar 

  • Ayyad S, Saleh A, Labib L (2019a) A new distributed feature selection technique for classifying gene expression data. Int J Biomath 12(2):1–34

    MathSciNet  MATH  Google Scholar 

  • Ayyad S, Saleh A, Labib L (2019b) Gene expression cancer classification using modified K-Nearest Neighbors technique. BioSystems 176:41–51

    Article  Google Scholar 

  • Bedi G, Venayagamoorthy K, Singh R, Brooks R, Wang K (2018) Review of internet of things (IoT) in electric power and energy systems. IEEE Internet Things J 5(2):847–870

    Article  Google Scholar 

  • Bera S, Misra S, Rodrigues J (2015) Cloud computing applications for smart grid: a survey. IEEE Trans Parallel Distrib Syst 26(5):1477–1494

    Article  Google Scholar 

  • Bhattarai B, Paudyal S, Luo Y et al (2019) Big data analytics in smart grids: state-of-the-art, challenges, opportunities, and future directions. IET Smart Grid, IET IEEE 2(2):141–154

    Article  Google Scholar 

  • Bikmetov R, Raja M, Sane T (2017) Infrastructure and applications of Internet of Things in smart grids: a survey. In: Proceedings of the 2017 North American Power Symposium (NAPS). IEEE, Morgantown, pp 1–6

  • Bourdeau M, Zhai X, Nefzaoui E, Guo X, Chatellier P (2019) Modelling and forecasting building energy consumption: a review of data-driven techniques. Sustain Cities Soc 48:1–27

    Article  Google Scholar 

  • Chomatek L, Duraj A (2019) Efficient genetic algorithm for breast cancer diagnosis. In: Proceedings of the International Conference on Information Technologies in Biomedicine, ITIB 2018: advances in intelligent systems and computing, vol 762, pp 64–76

  • Cui H, Peng X (2015) Short-term city electric load forecasting with considering temperature effects: an improved ARIMAX model. Math Probl Eng 2015:1–10

    MathSciNet  Google Scholar 

  • Daki H, El Hannani A, Aqqal A, Haidine A, Dahbi A (2017) Big Data management in smart grid: concepts, requirements and implementation. J Big Data 4(13):1–9

    Google Scholar 

  • Dinov I (2018) Probabilistic learning: classification using Naive Bayes. Data science and predictive analytics. Springer, Cham, pp 289–305

    Book  Google Scholar 

  • Ghallab H, Fahmy H, Nasr M (2019) Detection outliers on internet of things using big data technology. Egypt Inf J. https://www.sciencedirect.com/science/article/pii/S1110866519301616.

  • Ghanbari Z, Navimipour N, Hosseinzadeh M, Darwesh A (2019) A Resource allocation mechanisms and approaches on the Internet of Things. Cluster Comput 22(4):1253–1282

    Article  Google Scholar 

  • Ghasempour A (2019) Internet of Things in smart grid: architecture, applications, services, key technologies, and challenges. Inventions 4(22):1–12

    Google Scholar 

  • Hou W, Ning Z, Guo L, Zhang X (2019) Temporal, functional and spatial big data computing framework for large-scale smart grid. IEEE Trans Emerg Topics Comput 7(3):369–379

    Article  Google Scholar 

  • Houimli R, Zmami M, Ben-Salha O (2019) Short-term electric load forecasting in Tunisia using artificial neural networks. Energy Syst. https://doi.org/10.1007/s12667-019-00324-4

    Article  Google Scholar 

  • Hu J, Vasilakos A (2016) Energy big data analytics and security: challenges and opportunities. IEEE Trans Smart Grid 7(5):2423–2436

    Article  Google Scholar 

  • Hussain M, Beg M (2019) Fog Computing for Internet of Things (IoT)-Aided Smart Grid Architectures. Big Data Cogn Comput 3(8):1–29

    Google Scholar 

  • Jaradat M, Jarrah M, Bousselham A, Jararweh Y, Al-Ayyouba M (2015) The internet of energy: smart sensor networks and big data management for smart grid. Procedia Comput Sci 56:592–597

    Article  Google Scholar 

  • Jestes J (2013) Efficient summarization techniques for massive data. A thesis submitted to the faculty of the University of Utah in partial fulfillment of the requirements for the degree of Doctor of Philosophy, School of Computing, The University of Utah.

  • Jin C, Ma T, Hou R (2015) Chi-square statistics feature selection based on term frequency and distribution for text categorization. IETE J Res 61(4):351–362

    Article  Google Scholar 

  • Kabalci Y, Kabalci E, Padmanaban S, Holm-Nielsen J, Blaabjerg F (2019) Internet of things applications as energy internet in smart grids and smart environments. Electronics 8(972):1–16

    Google Scholar 

  • Kumari A, Tanwar S, Tyagi S et al (2019) Fog data analytics: a taxonomy and process model. J Netw Comput Appl 128:90–104

    Article  Google Scholar 

  • Liu X, Zhou Y, Chen X (2018) Mining outlier data in mobile internet-based large real-time databases. Complexity 2018:1–12. https://doi.org/10.1155/2018/9702304

    Article  Google Scholar 

  • Madhusudhanan B, Sumathi P, Karpagam N, Mahesh A, Suhi P (2019) An hybrid metaheuristic approach for efficient feature selection. Cluster Comput 22(9):14541–14549

    Article  Google Scholar 

  • Mangai J, Wagle S, Kumar V (2013) A novel web page classification model using an improved k nearest neighbor algorithm. In: Proceedings of the 3rd international conference on intelligent computational systems (ICICS’13), Singapore, pp 1–5

  • Manoj R, Praveena M, Vijayakumar K (2019) An ACO–ANN based feature selection algorithm for big data. Cluster Comput 22(2):3953–3960

    Article  Google Scholar 

  • Mao J, Wang T, Jin C, Zhou A (2017) Feature grouping-based outlier detection upon streaming trajectories. IEEE Trans Knowl Data Eng 29(12):2696–2709

    Article  Google Scholar 

  • Mary I, Arockiam L (2017) Detection of outliers in the IoT data using the STCPOD model. Int J Eng Res Comput Sci Eng 4(10):1–6

    Article  Google Scholar 

  • Marz N, Warren J (2015) Big data: principles and best practices of scalable realtime data systems. Shelter Island, Manning

    Google Scholar 

  • Moghaddass R, Wang J (2018) A hierarchical framework for smart grid anomaly detection using large-scale smart meter data. IEEE Trans Smart Grid 9(6):5820–5830

    Article  Google Scholar 

  • Motwani M, Tiwari A (2014) A novel semi supervised algorithm for text classification using BPNN by active search. IJCSI Int J Comput Sci Issues 11(3):154–160

    Google Scholar 

  • Mugunthan S, Vijayakumar T (2019) Review on IoT based smart grid architecture implementations. j Electric Eng Autom 1(1):12–20

    Article  Google Scholar 

  • Munshi A, Mohamed Y (2018) Data Lake Lambda architecture for smart grids big data analytics. IEEE Access IEEE 6:40463–40471

    Article  Google Scholar 

  • Nazar N, Senthilkumar R (2017) An online approach for feature selection for classification in big data. Turk J Electric Eng Comput Sci 25(1):163–171

    Article  Google Scholar 

  • Ou Q, Zhen Y, Li X, Zhang Y, Zeng L (2012) Application of Internet of Things in Smart Grid Power Transmission. In: Proceedings of the 2012 Third FTRA International Conference on Mobile, Ubiquitous, and Intelligent Computing, IEEE, Vancouver, BC, pp 96–100

  • Ozger M, Cetinkaya O, Akan O (2018) Energy harvesting cognitive radio networking for IoT-enabled smart grid. Mobile Netw Appl 23(4):956–966

    Article  Google Scholar 

  • Park C (2019) Outlier and anomaly pattern detection on data streams. J Supercomput 75(9):6118–6128

    Article  Google Scholar 

  • Qiang G (2010) An effective algorithm for improving the performance of Naïve Bayes for text classification. In: Proceedings of the 2010 second international conference on computer research and development. IEEE, Kuala Lumpur, pp 699–701

  • Rabie A, Saleh A, Abo-Al-Ez K (2015) A new strategy of load forecasting technique for smart grids. Int J Mod Trends Eng Res 2(12):332–341

    Google Scholar 

  • Rabie A, Ali S, Ali H, Saleh A (2019a) A fog based load forecasting strategy for smart grids using big electrical data. Cluster Comput 22(1):241–270

    Article  Google Scholar 

  • Rabie A, Ali S, Saleh A, Ali H (2019b) A new outlier rejection methodology for supporting load forecasting in smart grids based on big data. Cluster Comput. https://doi.org/10.1007/s10586-019-02942-0

    Article  Google Scholar 

  • Rabie A, Ali S, Saleh S, Ali H (2020) A fog based load forecasting strategy based on multi-ensemble classification for smart grids. J Ambient Intell Human Comput 11(1):209–236

    Article  Google Scholar 

  • Rahmani M, Atia G (2017) Randomized robust subspace recovery and outlier detection for high dimensional data matrices. IEEE Trans Signal Process 65(6):1580–1594

    Article  MathSciNet  MATH  Google Scholar 

  • Reddy S, Momoh J (2014) Short term electrical load forecasting using back propagation neural networks. In: Proceedings of the 2014 North American Power Symposium (NAPS). IEEE, Pullman, pp 1–6

  • Rong M, Gong D, Gao X (2019) Feature selection and its use in big data: challenges, methods, and trends. IEEE Access IEEE 7:19709–19725

    Article  Google Scholar 

  • Saeys Y, Inza I, Larranaga P (2007) A review of feature selection techniques in bioinformatics. Bioinformatics 23(19):2507–2517

    Article  Google Scholar 

  • Saleh A, Rabie A, Abo-Al-Ezb K (2016) A data mining based load forecasting strategy for smart electrical grids. Adv Eng Inform 30(3):422–448

    Article  Google Scholar 

  • Saneja B, Rani R (2019) A scalable correlation-based approach for outlier detection in wireless body sensor networks. Int J Commun Syst 32(7):1–15

    Article  Google Scholar 

  • Schuelke-Leech B, Barr B, Muratori M, Yurkovich B (2015) Big Data issues and opportunities for electric utilities. Renew Sustain Energy Rev 52:937–947

    Article  Google Scholar 

  • Shah S, Yaqoob I (2016) A Survey: Internet of Things (IoT) Technologies, Applications and Challenges. In: Proceedings of the 2016 IEEE Smart Energy Grid Engineering (SEGE). IEEE, Oshawa, pp 381–385

  • Sheng G, Hou H, Jiang X, Chen Y (2018) A novel association rule mining method of big data for power transformers state parameters based on probabilistic graph model. IEEE Trans Smart Grid 9(2):695–702

    Article  Google Scholar 

  • Shou Z, Li S (2018) Large dataset summarization with automatic parameter optimization and parallel processing for local outlier detection. Concurr Comput Pract Exp 30(23):1–13

    Article  Google Scholar 

  • Shuai C, Yang H, Ouyang X, He M, Gong Z, Shu W (2018) Analysis and identification of power blackout-sensitive users by using big data in the energy system. IEEE Access 7:19488–19501

    Article  Google Scholar 

  • Singh A, Ibraheem, Khatoon S, Muazzam M, Chaturvedi D (2012) Load forecasting techniques and methodologies: A review. In: Proceedings of the 2012 2nd International Conference on Power Control and Embedded Systems (ICPCES). Allahabad, pp 1–10

  • Singh G, Chauhan D, Chandel A (2017) Short-term load forecasting by using Ann, fuzzy logic and fuzzy neural network. Int J Eng Res Technol 6(1):384–389

    Google Scholar 

  • Sultan Y, Kaddah S, Elhosseini M (2017) Enhancing smart grid transient performance using storage devices-based MPC controller. IET Renew Power Gener 11(10):1316–1324

    Article  Google Scholar 

  • Sun G, Li J, Dai J, Song Z, Lang F (2018) MIC-based feature selection method for IoT data processing. Future Gener Comput Syst 89:606–616

    Article  Google Scholar 

  • Tellis V, Souza D (2018) Detecting anomalies in data stream using efficient techniques: a review. In: Proceedings of the 2018 International Conference on Control, Power, Communication and Computing Technologies (ICCPCCT). Kannur, pp 296–298

  • Torabi A, Mousavy S, Dashti V, Saeedi M, Yousefi N (2019) A new prediction model based on cascade NN for wind power prediction. Comput Econ 53(3):1219–1243

    Article  Google Scholar 

  • Tu C, He X, Shuai Z, Jiang F (2017) Big data issues in smart grid – A review. Renew Sustain Energy Rev 79:1099–1107

    Article  Google Scholar 

  • Vasconcelos I, Vasconcelos R, Olivieri B, Roriz M, Endler M, Junior M (2017) Smartphone-based outlier detection: a complex event processing approach for driving behavior detection. J Internet Serv Appl 8(13):1–30

    Google Scholar 

  • VeeraManickam M, Mohanapriya M, Pandey B, Akhade S, Kale S, Patil R, Vigneshwar M (2019) Map-Reduce framework based cluster architecture for academic student’s performance prediction using cumulative dragonfly based neural network. Cluster Comput 22(1):1259–1275

    Article  Google Scholar 

  • Venkatesh G, Arunesh K (2019) Map Reduce for big data processing based on traffic aware partition and aggregation. Cluster Comput 12(5):12909–12915

    Article  Google Scholar 

  • Vimala S, Sharmili K (2018) Prediction of loan risk using Naive Bayes and support vector machine. Int Conf Adv Comput Technol 4(2):110–113

    Google Scholar 

  • Vimalkumar K, Radhika N (2017) A big data framework for intrusion detection in smart grids using Apache Spark. In: Proceedings of the 2017 International conference on advances in computing, communications and informatics (ICACCI). IEEE, pp 198–204

  • Wang K, Xu C, Zhang Y, Guo S, Zomaya A (2019) Robust big data analytics for electricity price forecasting in the smart grid. IEEE Trans Big Data IEEE 5(1):34–45

    Article  Google Scholar 

  • Wang Y, Ke W, Tao X (2016) A feature selection method for large-scale network traffic classification based on Spark. Information 7(1):1–11. www.mdpi.com/journal/information

  • Wosiak A, Zakrzewska D (2018) Integrating correlation-based feature selection and clustering for improved cardiovascular disease diagnosis. Complexity Hindawi 2018:1–11

    Article  Google Scholar 

  • Yan Y, Qian Y, Sharif H, Tipper D (2013) A survey on smart grid communication infrastructures: motivations, requirements and challenges. IEEE Commun Surv Tutor 15(1):5–20

    Article  Google Scholar 

  • Yan Y, Cao L, Rundensteiner E (2017) Distributed Top-N Local Outlier Detection in Big Data. In: Proceedings of the 2017 IEEE International Conference on Big Data (Big Data). Boston, pp 827–836

  • Yang C, Chen S, Liu J, Liu R, Chang C (2019) On construction of an energy monitoring service using big data technology for the smart campus. Cluster Comput. https://doi.org/10.1007/s10586-019-02921-5

    Article  Google Scholar 

  • Yu W, Liang F, He X, Hatcher W, Lu C, Lin J, Yang X (2018) A survey on the edge computing for the internet of things. IEEE Access 6:6900–6919

    Article  Google Scholar 

  • Zahid M, Ahmed F, Javaid N, Abbasi R, Kazmi H, Javaid A, Bilal M, Akbar M, Ilahi M (2019) Electricity price and load forecasting using enhanced convolutional neural network and enhanced support vector regression in smart Grids. Electronics 8(2):1–32

    Article  Google Scholar 

  • Zdravevski E, Lameski P, Kulakov A, Jakimovski B, Filiposka S, Trajanov D (2015) Feature ranking based on information gain for large classification problems with MapReduce. In: Proceedings of the 2015 IEEE Trustcom/BigDataSE/ISPA. IEEE, Helsinki, pp 186–191

  • Zhang S, Li X, Zong M, Zhu X, Wang R (2018a) Efficient kNN classification with different numbers of nearest neighbors. IEEE Trans Neural Netw Learn Syst 29(5):1774–1784

    Article  MathSciNet  Google Scholar 

  • Zhang Y, Huang T, Bompard E (2018b) Big data analytics in smart grids: a review. Energy Inf 1(8):1–24

    Google Scholar 

  • Zhou K, Fu C, Yang S (2016) Big data driven smart energy management: from big data to big insights. Renew Sustain Energy Rev 56:215–225

    Article  Google Scholar 

  • Zhukov A, Tomin N, Kurbatsky V, Sidorov D, Panasetsky D, Foley A (2019) Ensemble methods of classification for power systems security assessment. Appl Comput Inf 15(1):45–53

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computers and Systems Department, Faculty of Engineering, Mansoura University, Mansoura, Egypt

    Asmaa H. Rabie, Ahmed I. Saleh & Hesham A. Ali

Authors
  1. Asmaa H. Rabie
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ahmed I. Saleh
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hesham A. Ali
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Asmaa H. Rabie.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rabie, A.H., Saleh, A.I. & Ali, H.A. Smart electrical grids based on cloud, IoT, and big data technologies: state of the art. J Ambient Intell Human Comput 12, 9449–9480 (2021). https://doi.org/10.1007/s12652-020-02685-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12652-020-02685-6

Keywords

Search

Navigation