Skip to main content
Fachgebiete
Automobil + Motoren Bauwesen + Immobilien Business IT + Informatik Elektrotechnik + Elektronik Energie + Nachhaltigkeit Finance + Banking Management + Führung Marketing + Vertrieb Maschinenbau + Werkstoffe Versicherung + Risiko
DE
EN
Bücher
Zeitschriften
Themenseiten
Organisationspsychologie Projektmanagement Marketing Smart Manufacturing
Weitere Formate
Podcasts Webinare Technik Webinare Wirtschaft Kongresse Veranstaltungskalender Awards
Jetzt Einzelzugang starten
Zugang für Unternehmen
Referenzkunden
SLX-Digitalkonferenz: Zukunftswerkstatt 2024

Mehr Umsatz mit Top-Kontakten

Am 7. Mai erfahren Sie, wie Vertriebsteams Leadgenerierung, Leadnurturing und Leadstrategien effizient steuern können.
Erweiterte Suche
Anmelden
nach oben
Journal of Computer and Systems Sciences International
Erschienen in: Journal of Computer and Systems Sciences International 6/2020

01.11.2020 | ARTIFICIAL INTELLIGENCE

The Concept of Constructing an Artificial Dispatcher Intelligent System Based on Deep Reinforcement Learning for the Automatic Control System of Electric Networks

verfasst von: N. V. Tomin

Erschienen in: Journal of Computer and Systems Sciences International | Ausgabe 6/2020

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

The concept of the Artificial Dispatcher autonomous intelligent control system for integrating into modern automatic control systems of electric networks in order to improve the efficiency of managing the modes of electric power systems is proposed. This intelligent control system is implemented based on the deep machine learning technology and joint use of the Monte Carlo tree search method and deep artificial neural networks. It is shown that the efficient training of the Artificial Dispatcher intelligent control system is achievable through an innovative self-play algorithm, without using large databases of circuit-mode situations and expertise in managing the modes of electric power systems. Calculation examples of using agents created based on the concept of the Artificial Dispatcher intelligent control system in the problems of the automation of substation technological processes (managing voltage and reactive power controllers) and industrial enterprises (DC servomotor control) are presented.
Vorheriger Artikel Recursive Definitions of Tabular Transformations
Nächster Artikel Synthesis-Analysis of Derivation Sequences of Tables’ Functional Dependences

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Jetzt informieren
Fußnoten
1
Statistical process control is a method of monitoring the production process using statistical tools for the purpose of product quality control “directly in-process.”
 
2
Considering that the RL methods are based on the MDMP theory, when the strategy of control is optimal, if it achieves the best (maximum) expected reward from any state [30], the initial statistical distributions for functions V(s) or Q(s,a) do not play any role. Although formally, the maximization of a state value function for an unknown initial distribution would require V(s) = \(E\left[ {R({{s}_{0}},{{a}_{0}}) + {{\gamma }}R({{s}_{1}},{{a}_{1}}) + \ldots |{{\pi }}} \right]\) [31].
 
3
Superhuman performance is a term introduced to evaluate the actions of machine intelligence that lead to more efficient problem-solving in the subject area than the expert would have achieved [37].
 
Literatur
1.
A. D. Drozdov, A. S. Zasypkin, A. A. Alliluev, et al., Automation of Energy Systems, The School-Book (Moscow, Energiya, 1977) [in Russian].
2.
“The world market study of SCADA, energy management systems, distribution management systems and outage management systems in electric utilities: 2017–2019,” 4-Volume Report (Newton-Evans Research Comp., Dublin, 2017).
3.
N. I. Voropai, N. V. Tomin, D. N. Sidorov, et al., “A suite of intelligent tools for early detection and prevention of blackouts in power interconnections,” Autom Remote Control. 79, 1741–1755 (2018).CrossRef
4.
D. A. Panasetskii, “Improving the structure and algorithms of emergency control of EPSs to prevent an avalanche of voltage and cascade shutdown of lines,” Cand. Sci. (Tech. Sci.) Dissertation (Melentiev Energy Syst. Inst. Siber. Branch of RAS, Irkutsk, 2015).
5.
A. M. Suleimanova, “Intelligent advisor disaster management electricity system,” Cand. Sci. (Tech. Sci.) Dissertation (Ufa State Aviation Tech. Univ., Ufa, 1993).
6.
D. A. Pospelov, “Principles of situation management,” Izv. Akad. Nauk SSSR, Tekh. Kibern., No. 2, 3–10 (1971).
7.
I. V. Kirilin, “Classification of states of electric networks of industrial enterprises for reactive power compensation control,” Cand. Sci. (Tech. Sci.) Dissertation (Sib. Fed. Univ., Krasnoyarsk, 2011).
8.
T. Hiyama, W. Hubbi, and T. H. Ortmeyer, “Fuzzy logic control scheme with variable gain for static var compensator to enhance power system stability,” IEEE Trans. Power Syst. 4, 186–191 (1999).CrossRef
9.
Cascade-NT 2.0 Complex. http://​www.​cascadent.​ru/​cascade2016.​pdf.​
10.
Volt-VAr Management Solutions for Smart Grid Distribution Automation Applications. Product Brochure of the ABB Smart Grid Center of Excellence. https://library.e.abb.com/public/d9e73a8d3d91161bc1257b6a006cc340/VVMS%20brochure_final_v4.pdf.
11.
S. P. Grigoryev, Automated Process Control System: Errors of the First and Second Kind, Deming Pro. www.deming.pro/spc-cases-apcs.html.
12.
M. Negnevitsky, Artificial Intelligence: A Guide to Intelligent Systems, 3rd ed. (Addison Wesley, Harlow, UK, 2011).
13.
A. Kaci, I. Kamwa, L. Dessaint, et al., “Synchrophasor data baselining and mining for online monitoring of dynamic security limits,” IEEE Trans. Power Syst. 29, 2681–2695 (2014).CrossRef
14.
C. Liu, C. L. Bak, Z. Chen, et al., “Dynamic security assessment of western Danish power system based on ensemble decision trees,” in Proceedings of the 12th IET International Conference on Developments in Power System Protection, Copenhagen, 2014.
15.
N. V. Tomin, V. G. Kurbatsky, and I. S. Reutsky, “Hybrid intelligent technique for voltage/VAR control in power systems,” IET Generat. Transmiss. Distrib. 13, 4724–4732 (2019).CrossRef
16.
I. Kogan, K. Boehme, and H. J. Herrmann, “Introducing advanced machine learning technology in protection relays,” in Proceedings of the Conference and Exhibitions on Relay Protection and Automation of Power Systems 2017, St. Petersburg, 2017.
17.
R. Busch, “The future of manufacturing. artificial intelligence: Optimizing industrial operations,” Siemens Web Page (2018). www.siemens.com/innovation/en/home/pictures-of-the-future/industry-and-automation/the-future-of-manufacturing-ai-in-industry.html.
18.
S. Webel, K. Nikolaus, and A. F. Pease, “Autonomous systems. getting machines to mimic intuition,” Siemens Web Page (2016). www.siemens.com/innovation/en/home/pictures-of-the-future/digitalization-and-software/autonomous-systems-machine-learning.html.
19.
E. Mocanu, P. H. Nguyen, and M. Gibescu, “Deep learning for power system data analysis,” in Big Data Application in Power Systems, Ed. by R. Arghandeh and Y. Zhou (Elsevier, Amsterdam, 2018), pp. 125–158.
20.
Y. Tang, H. He, J. Wen, and J. Liu, “Power system stability control for a wind farm based on adaptive dynamic programming,” IEEE Trans. Smart Grid 6, 166–177 (2015).CrossRef
21.
Y. Liu, “Machine learning for wind power prediction,” Master’s Thesis (Univ. New Brunswick, Canada, 2016).
22.
V. Francois-Lavet, D. Taralla, D. Ernst, et al., “Deep reinforcement learning solutions for energy microgrids management,” in Proceedings of the European Workshop on Reinforcement Learning, Barselona, 2016.
23.
A. Zhukov, N. Tomin, D. Sidorov, V. Kurbatsky, and D. Panasetsky, “On-line power systems security assessment using data stream random forest algorithm,” in Innovative Computing, Optimization and Its Applications, Vol. 741 of Studies in Computational Intelligence, Ed. by I. Zelinka, P. Vasant, V. Duy, and T. Dao (Springer, New York, 2018).
24.
N. Tomin, M. Negnevitsky, and Ch. Rehtanz, “Preventing large-scale emergencies in modern power systems: AI approach,” Adv. Comput. Intell. Intell. Inform. 18, 714–727 (2014).CrossRef
25.
Y. Xu, W. Zhang, W. Liu, and F. Ferrese, “Multiagent-based reinforcement learning for optimal reactive power dispatch,” IEEE Trans. Syst., Man, Cybern. 42, 1742–1751 (2012).CrossRef
26.
R. Belkacemi, A. Abdulrasheed Babalola, and S. Zarrabian, “Real-time cascading failures prevention through MAS algorithm and immune system reinforcement learning,” Electric Power Compon. Syst. 45, 505–519 (2017).CrossRef
27.
D. Ye, M. Zhang, and D. Sutanto, “A hybrid multiagent framework with q-learning for power grid systems restoration,” IEEE Trans. Power Syst. 26, 2434–2441 (2011).CrossRef
28.
S. Zarabbian, R. Belkacemi, and A. A. Babalola, “Reinforcement learning approach for congestion management and cascading failure prevention with experimental application,” Electr. Power Syst. Res. 141, 179–190 (2016).CrossRef
29.
M. Glavic, D. Ernst, and L. Wehenkel, “A reinforcement learning based discrete supplementary control for power system transient stability enhancement,” Eng. Intell. Syst. Electr. Eng. Commun. 13, 81–88 (2005).
30.
R. S. Sutton and E. G. Barto, Reinforcement Learning: An Introduction (MIT Press, Cambridge, MA, 2014).MATH
31.
M. El Chamie and B. Acikmese, “Finite-horizon  Markov decision processes with state constraints,” arXiv:1507.01585 [math.OC] (2015).
32.
M. Glavic, R. Fonteneau, and D. Ernst, “Reinforcement learning for electric power system decision and control: Past considerations and perspectives,” IFAC-PapersOnLine 50, 6918–6927 (2017).CrossRef
33.
R. Yousefian and K. Kamalasadan, “Design and real-time implementation of optimal power system wide-area system-centric controller based on temporal difference learning,” IEEE Trans. Industry Appl. 1, 395–401 (2015).
34.
D. Ernst, L. Wehenkel, and M. Glavic, “Power systems stability control: Reinforcement learning framework,” IEEE Trans. Power Syst. 19, 427–435 (2004).CrossRef
35.
D. Ernst, M. Glavic, F. Capitanescu, et al., “Reinforcement learning versus model predictive control: A comparison on a power system problem,” IEEE Trans. Syst., Man, Cybern. 39, 517–529 (2009).CrossRef
36.
S. Vandael, B. Claessens, D. Ernst, et al., “Reinforcement learning of heuristic EV fleet charging in a day-ahead electricity market,” IEEE Trans. Smart Grid 6, 1795–1805 (2015).CrossRef
37.
D. Silver, J. Schrittwieser, K. Simonyan, et al., “Mastering the game of go without human knowledge,” Nature (London, U.K.) 550, 354–359 (2017).CrossRef
38.
V. Mnih, K. Kavukcuoglu, D. Silver, et al., “Playing atari with deep reinforcement learning,” arXiv:1312.5602.
39.
A. Irpan, “Deep reinforcement learning doesn’t work yet,” Sorta Insightful (2018). www.alexirpan.com/2018/02/14/rl-hard.html.
40.
D. Silver, A. Huang, Ch. Maddison, et al., “Mastering the game of go with deep neural networks and tree search,” Nature (London, U.K.) 529 (7587), 484–489 (2016).CrossRef
41.
Software Package for Creating Automated Systems for Operational-Technological Management in Network Companies SK-11. http://​www.​monitel.​ru/​files/​downloads/​products/​Broshyura%20-%20CK-11.pdf?201711.
42.
N. V. Tomin, V. G. Kurbatsky, and M. Negnevitsky, “The concept of the deep learning-based system artificial dispatcher to power system control and dispatch,” arXiv:1805.05408 [cs.CY] (2018).
43.
F. Milano, Power System Modelling and Scripting (Springer, London, 2010).CrossRef
44.
ANARES Software and Computer Complex. http://​anares.​ru.​
45.
N. I. Voropai, N. V. Tomin, V. G. Kurbatskii, et al., A Set of Intelligent Tools to Prevent Major Accidents in Power Systems (Nauka, Novosibirsk, 2016) [in Russian].
46.
F. Li and Yan Du, “From AlphaGo to power system AI: What engineers can learn from solving the most complex board game,” IEEE Power Energy Mag. 16 (2), 76–84 (2018).CrossRef
47.
M. Egorov, Z. H. Sunberg, E. Balaban, et al., “POMDPs.Jl: A framework for sequential decision making under uncertainty,” Machine Learning Res. 18 (26), 1–5 (2017).MathSciNet
48.
M. Patacchiola, Dissecting Reinforcement Learning, Github. https://​github.​com/​mpatacchiola/​dissecting-reinforcement-learning.​
49.
L. Sheng, “Strategy research of substation voltage reactive control basing on the ninth region plot,” Qinghai Electr. Power 24, 1–4 (2005).
50.
X. Wu, J.-Ch. Wang, P. Yang, et al., “Fuzzy control on voltage/reactive power in electric power substation,” in Fuzzy Information and Engineering, Proceedings of the 3rd International Conference on Fuzzy Information and Engineering (ICFIE 2009), Chongqing, China, September 26–29, 2009, Ed. by B. Cao, T.-F. Li, and C.-Y. Zhang, Vol. 2, pp. 1083–1091.
51.
R. Manikandan and R. Arulmozhiyal, “Position control of DC servo drive using fuzzy logic controller,” in Proceedings of the International Conference on Advances in Electrical Engineering (ICAEE), India, 2014.
52.
H. Yunhai, X. Bingfeng, and L. Gen Ping, “A power control method for inverted pendulum based on fuzzy control,” in Proceedings of the International Conference on Computer, Mechatronics, Control and Electronic Engineering, China, 2010.
Metadaten
Titel
The Concept of Constructing an Artificial Dispatcher Intelligent System Based on Deep Reinforcement Learning for the Automatic Control System of Electric Networks
verfasst von
N. V. Tomin
Publikationsdatum
01.11.2020
Verlag
Pleiades Publishing
Erschienen in
Journal of Computer and Systems Sciences International / Ausgabe 6/2020
Print ISSN: 1064-2307
Elektronische ISSN: 1555-6530
DOI
https://doi.org/10.1134/S1064230720050111

Weitere Artikel der Ausgabe 6/2020

Journal of Computer and Systems Sciences International 6/2020 Zur Ausgabe

COMPUTER METHODS

Software for the Automated Control of Robotic Optical Observatories

ARTIFICIAL INTELLIGENCE

Recursive Definitions of Tabular Transformations

MATHEMATICAL SIMULATION

Search-Detection-Recognition: Simulation via Thermal Images with Varying Quality

STABILITY

Stability of an Interval Family of Differential-Algebraic Equations

CONTROL IN STOCHASTIC SYSTEMS AND UNDER UNCERTAINTY CONDITIONS

An Optimal Recurrent Logical-Dynamical Filter of a High Order and Its Covariance Approximations

DATA PROCESSING AND IDENTIFICATION

Algebraic Method of Nonparametric Identification of Abnormal Modes of the Power System as a Dynamic MIMO System

Premium Partner

    Bildnachweise