Skip to main content

2018 | OriginalPaper | Buchkapitel

Using Reinforcement Learning to Handle the Runtime Uncertainties in Self-adaptive Software

verfasst von : Tong Wu, Qingshan Li, Lu Wang, Liu He, Yujie Li

Erschienen in: Software Technologies: Applications and Foundations

Verlag: Springer International Publishing

Aktivieren Sie unsere intelligente Suche um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

The growth of scale and complexity of software as well as the complex environment with high dynamic lead to the uncertainties in self-adaptive software’s decision making at run time. Self-adaptive software needs the ability to avoid negative effects of uncertainties to the quality attributes of the software. However, existing planning methods cannot handle the two types of runtime uncertainties caused by complexity of system and running environment. This paper proposes a planning method to handle these two types of runtime uncertainties based on reinforcement learning. To handle the uncertainty from the system, the proposed method can exchange ineffective self-adaptive strategies to effective ones according to the iterations of execution effects at run time. It can plan dynamically to handle uncertainty from environment by learning knowledge of relationship between system states and actions. This method can also generate new strategies to deal with unknown situations. Finally, we use a complex distributed e-commerce system, Bookstore, to validate the ability of proposed method.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Krupitzer, C., Roth, F.M., Vansyckel, S., et al.: A survey on engineering approaches for self-adaptive systems. Pervasive Mob. Comput. 17(PB), 184–206 (2015)CrossRef Krupitzer, C., Roth, F.M., Vansyckel, S., et al.: A survey on engineering approaches for self-adaptive systems. Pervasive Mob. Comput. 17(PB), 184–206 (2015)CrossRef
2.
Zurück zum Zitat Cheng, S.W., Garlan, D.: Handling uncertainty in autonomic systems. In: International Workshop on Living with Uncertainties (2007) Cheng, S.W., Garlan, D.: Handling uncertainty in autonomic systems. In: International Workshop on Living with Uncertainties (2007)
3.
Zurück zum Zitat Esfahani, N., Kouroshfar, E., Malek, S., et al.: Taming uncertainty in self-adaptive software. In: 13th European conference on Foundations of Software Engineering, pp. 234–244. ACM (2011) Esfahani, N., Kouroshfar, E., Malek, S., et al.: Taming uncertainty in self-adaptive software. In: 13th European conference on Foundations of Software Engineering, pp. 234–244. ACM (2011)
4.
Zurück zum Zitat Elkhodary, A., Esfahani, N., Malek, S., et al.: FUSION: a framework for engineering self-tuning self-adaptive software systems. In: 18th ACM SIGSOFT International Symposium on Foundations of Software Engineering, pp. 7–16. ACM (2010) Elkhodary, A., Esfahani, N., Malek, S., et al.: FUSION: a framework for engineering self-tuning self-adaptive software systems. In: 18th ACM SIGSOFT International Symposium on Foundations of Software Engineering, pp. 7–16. ACM (2010)
5.
Zurück zum Zitat Mao, X., Dong, M., Liu, L., et al.: An integrated approach to developing self-adaptive software. J. Inf. Sci. Eng. 30(4), 1071–1085 (2014) Mao, X., Dong, M., Liu, L., et al.: An integrated approach to developing self-adaptive software. J. Inf. Sci. Eng. 30(4), 1071–1085 (2014)
6.
Zurück zum Zitat Amoui, M., Salehie, M., Mirarab, S., et al.: Adaptive action selection in autonomic software using reinforcement learning. In: International Conference on Autonomic and Autonomous Systems, pp. 175–181. IEEE Computer Society (2008) Amoui, M., Salehie, M., Mirarab, S., et al.: Adaptive action selection in autonomic software using reinforcement learning. In: International Conference on Autonomic and Autonomous Systems, pp. 175–181. IEEE Computer Society (2008)
Metadaten
Titel
Using Reinforcement Learning to Handle the Runtime Uncertainties in Self-adaptive Software
verfasst von
Tong Wu
Qingshan Li
Lu Wang
Liu He
Yujie Li
Copyright-Jahr
2018
DOI
https://doi.org/10.1007/978-3-030-04771-9_28