Skip to main content

2019 | OriginalPaper | Buchkapitel

SPI: A Software Tool for Planning Under Uncertainty Based on Learning Factored MDPs

verfasst von : Alberto Reyes, Pablo H. Ibargüengoytia, Guillermo Santamaría

Erschienen in: Advances in Soft Computing

Verlag: Springer International Publishing

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

In this paper the SPI system is presented. SPI is a software tool for planning under uncertainty based on learning Markov Decision Processes. A brief review of some similar tools as well as the scientific basis of factored representations and some of its variants are included. Among these variants are qualitative representations and hybrid qualitative-discrete representations that are the core of the software tool. The functional structure of SPI, which is composed of four main modules, is also described. These modules are: the compiler, the policy server, a format translator and a didactic simulator. The experimental results obtained when testing SPI in a robot navigation domain using different types of representations and different state partitions demonstrated its capability to reduce state spaces.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Bellman, R.E.: Dynamic Programming. Princeton University Press, Princeton (1957)MATH Bellman, R.E.: Dynamic Programming. Princeton University Press, Princeton (1957)MATH
2.
Zurück zum Zitat Chadès, I., Chapron, G., Cros, M.J., Garcia, F., Sabbadin, R.: MDPtoolbox: a multi-platform toolbox to solve stochastic dynamic programming problems. Ecography 37, 916–920 (2014)CrossRef Chadès, I., Chapron, G., Cros, M.J., Garcia, F., Sabbadin, R.: MDPtoolbox: a multi-platform toolbox to solve stochastic dynamic programming problems. Ecography 37, 916–920 (2014)CrossRef
3.
Zurück zum Zitat Cooper, G.F., Herskovits, E.: A Bayesian method for the induction of probabilistic networks from data. Mach. Learn. 9(4), 309–347 (1992)MATH Cooper, G.F., Herskovits, E.: A Bayesian method for the induction of probabilistic networks from data. Mach. Learn. 9(4), 309–347 (1992)MATH
4.
Zurück zum Zitat Hoey, J., St-Aubin, R., Hu, A., Boutilier, C.: SPUDD: stochastic planning using decision diagrams. In: Proceedings of the 15th Conference on Uncertainty in AI, UAI 1999, pp. 279–288 (1999) Hoey, J., St-Aubin, R., Hu, A., Boutilier, C.: SPUDD: stochastic planning using decision diagrams. In: Proceedings of the 15th Conference on Uncertainty in AI, UAI 1999, pp. 279–288 (1999)
5.
Zurück zum Zitat Munos, R., Moore, A.: Variable resolution discretization for high-accuracy solutions of optimal control problems. In: Dean, T. (ed.) Proceedings of the 16th International Joint Conference on Artificial Intelligence (IJCAI 1999), pp. 1348–1355. Morgan Kaufmann Publishers, San Francisco (1999) Munos, R., Moore, A.: Variable resolution discretization for high-accuracy solutions of optimal control problems. In: Dean, T. (ed.) Proceedings of the 16th International Joint Conference on Artificial Intelligence (IJCAI 1999), pp. 1348–1355. Morgan Kaufmann Publishers, San Francisco (1999)
6.
Zurück zum Zitat Porta, J.M., Vlassis, N., Spaan, M.T.J., Poupart, P.: Point-based value iteration for continuous POMDPs. J. Mach. Learn. Res. 7, 2329–2367 (2006)MathSciNetMATH Porta, J.M., Vlassis, N., Spaan, M.T.J., Poupart, P.: Point-based value iteration for continuous POMDPs. J. Mach. Learn. Res. 7, 2329–2367 (2006)MathSciNetMATH
7.
Zurück zum Zitat Poupart, P.: Exploiting structure to efficiently solve large scale partially observable Markov decision processes. Ph.D. thesis, University of Toronto (2005) Poupart, P.: Exploiting structure to efficiently solve large scale partially observable Markov decision processes. Ph.D. thesis, University of Toronto (2005)
8.
9.
Zurück zum Zitat Quinlan, J.: C4.5: Programs for Machine Learning. Morgan Kaufmann, San Francisco (1993) Quinlan, J.: C4.5: Programs for Machine Learning. Morgan Kaufmann, San Francisco (1993)
10.
Zurück zum Zitat Reyes, A., Sucar, L.E., Morales, E., Ibarguengoytia, P.H.: Abstraction and refinement for solving Markov decision processes. In: Workshop on Probabilistic Graphical Models, PGM 2006, Chezch Republic, pp. 263–270 (2006) Reyes, A., Sucar, L.E., Morales, E., Ibarguengoytia, P.H.: Abstraction and refinement for solving Markov decision processes. In: Workshop on Probabilistic Graphical Models, PGM 2006, Chezch Republic, pp. 263–270 (2006)
11.
Zurück zum Zitat Reyes, A., Sucar, L.E., Morales, E.F.: AsistO: a qualitative MDP-based recommender system for power plant operation. Computacion y Sistemas 13(1), 5–220 (2009) Reyes, A., Sucar, L.E., Morales, E.F.: AsistO: a qualitative MDP-based recommender system for power plant operation. Computacion y Sistemas 13(1), 5–220 (2009)
13.
Zurück zum Zitat Sandoval, C., Galindo, X., Salas, R.: Herramienta software para resolver procesos de decisiÃn de Markov utilizando recocido simulado. In: Memorias de la Décima Quinta Conferencia Iberoamericana en Sistemas, Cibernética e Informática (CISCI 2016) (2016) Sandoval, C., Galindo, X., Salas, R.: Herramienta software para resolver procesos de decisiÃn de Markov utilizando recocido simulado. In: Memorias de la Décima Quinta Conferencia Iberoamericana en Sistemas, Cibernética e Informática (CISCI 2016) (2016)
14.
Zurück zum Zitat Sarmiento, A., Riaño, G.: JMDP: an object oriented framework for modeling MDPs. In: Informatics Annual Meeting (2006) Sarmiento, A., Riaño, G.: JMDP: an object oriented framework for modeling MDPs. In: Informatics Annual Meeting (2006)
15.
Zurück zum Zitat Sigaud, O., Buffet, O.: Markov Decision Processes in Artificial Intelligence. ISTE Ltd./Wiley, London/Hoboken (2010)MATH Sigaud, O., Buffet, O.: Markov Decision Processes in Artificial Intelligence. ISTE Ltd./Wiley, London/Hoboken (2010)MATH
Metadaten
Titel
SPI: A Software Tool for Planning Under Uncertainty Based on Learning Factored MDPs
verfasst von
Alberto Reyes
Pablo H. Ibargüengoytia
Guillermo Santamaría
Copyright-Jahr
2019
DOI
https://doi.org/10.1007/978-3-030-33749-0_38