Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
Published in:
Cover of the book

2015 | OriginalPaper | Chapter

A Hybrid POMDP-BDI Agent Architecture with Online Stochastic Planning and Desires with Changing Intensity Levels

Authors : Gavin Rens, Thomas Meyer

Published in: Agents and Artificial Intelligence

Publisher: Springer International Publishing

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

We propose an agent architecture which combines Partially observable Markov decision processes (POMDPs) and the belief-desire-intention (BDI) framework to capitalize on their complimentary strengths. Our architecture introduces the notion of intensity of the desire for a goal’s achievement. We also define an update rule for goals’ desire levels. When to select a new goal to focus on is also defined. To verify that the proposed architecture works, experiments were run with an agent based on the architecture, in a domain where multiple goals must continually be achieved. The results show that (i) while the agent is pursuing goals, it can concurrently perform rewarding actions not directly related to its goals, (ii) the trade-off between goals and preferences can be set effectively and (iii) goals and preferences can be satisfied even while dealing with stochastic actions and perceptions. We believe that the proposed architecture furthers the theory of high-level autonomous agent reasoning.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now
next chapter Dynamic JChoc: A Distributed Constraints Reasoning Platform for Dynamically Changing Environments
Footnotes
1
\( Pref (\cdot )\) is designed such that the agent collects a maximum number of items (ignoring goals). The agent collects more when it is encouraged to sense where items are, hence \( sensUtil \) is 1 if the agent tries to \( see \).
 
2
Essentially, the goals in G are stacked in descending order of the value of \(V^*_ HPB (B,g,h^-)\), where \(h^- < h\) and B is the current belief-state. The goal on top of the stack becomes the intention.
 
Literature
1.
Bratman, M.: Intention, Plans, and Practical Reason. Harvard University Press, Massachusetts (1987)
2.
Rao, A., Georgeff, M.: BDI agents: From theory to practice. In: Proceedings of the ICMAS 1995, pp. 312–319. AAAI Press (1995)
3.
Monahan, G.: A survey of partially observable Markov decision processes: theory, models, and algorithms. Manage. Sci. 28, 1–16 (1982)MathSciNetCrossRefMATH
4.
5.
Koenig, S.: Agent-centered search. Artif. Intell. Mag. 22, 109–131 (2001)
6.
Ross, S., Pineau, J., Paquet, S., Chaib-draa, B.: Online planning algorithms for POMDPs. J. Artif. Intell. Res. (JAIR) 32, 663–704 (2008)MathSciNetMATH
7.
Schut, M., Wooldridge, M., Parsons, S.: The theory and practice of intention reconsideration. Exp. Theor. Artif. Intell. 16, 261–293 (2004)CrossRef
8.
Wooldridge, M.: Intelligent agents. In: Weiss, G. (ed.) Multiagent Systems: A Modern Approach to Distributed Artificial Intelligence. MIT Press, Massachusetts (1999)
9.
Wooldridge, M.: An Introduction to Multiagent Systems. Wiley, Chichester (2002)
10.
Wooldridge, M.: Reasoning About Rational Agents. MIT Press, Massachusetts (2000)MATH
11.
Schut, M., Wooldridge, M.: Principles of intention reconsideration. In: Agents 2001: Proceedings of the 5th International Conference on Autonomous Agents, pp. 340–347. ACM Press, New York (2001)
12.
Pollack, M., Ringuette, M.: Introducing the tileworld: experimentally evaluating agent architectures. In: Proceedings of the AAAI 1990, pp. 183–189. AAAI Press (1990)
13.
Kinny, D., Georgeff, M.: Commitment and effectiveness of situated agents. In: Proceedings of the 12th International Joint Conference on Artificial Intelligence (IJCAI-91), pp. 82–88 (1991)
14.
Kinny, D., Georgeff, M.: Experiments in optimal sensing for situated agents. In: Proceedings of the 2nd Pacific Rim Internatioanl Conference on Artificial Intelligence (PRICAI 1992) (1992)
15.
Schut, M., Wooldridge, M.: Intention reconsideration in complex environments. In: Proceedings of the 4th International Conference on Autonomous Agents (AGENTS 2000). ACM, New York (2000)
16.
Schut, M., Wooldridge, M.: The control of reasoning in resource-bounded agents. Knowl. Eng. Rev. 16, 215–240 (2001)CrossRef
17.
Kaelbling, L., Littman, M., Cassandra, A.: Planning and acting in partially observable stochastic domains. Artif. Intell. 101, 99–134 (1998)MathSciNetCrossRefMATH
18.
Walczak, A., Braubach, L., Pokahr, A., Lamersdorf, W.: Augmenting BDI agents with deliberative planning techniques. In: Bordini, R.H., Dastani, M., Dix, J., El Fallah Seghrouchni, A. (eds.) PROMAS 2006. LNCS (LNAI), vol. 4411, pp. 113–127. Springer, Heidelberg (2007) CrossRef
19.
Meneguzzi, F., Zorzo, A., Móra, M., Luck, M.: Incorporating planning into BDI systems. Scalable Comput. Pract. Experience 8, 15–28 (2007)
20.
Nair, R., Tambe, M.: Hybrid bdi-pomdp framework for multiagent teaming. J. Artif. Intell. Res. (JAIR) 23, 367–420 (2005)MATH
21.
Lim, M.Y., Dias, J., Aylett, R.S., Paiva, A.C.R.: Improving adaptiveness in autonomous characters. In: Prendinger, H., Lester, J.C., Ishizuka, M. (eds.) IVA 2008. LNCS (LNAI), vol. 5208, pp. 348–355. Springer, Heidelberg (2008) CrossRef
22.
Pereira, D., Gonçalves, L., Dimuro, G., Costa, A.: Constructing bdi plans from optimal pomdp policies, with an application to agentspeak programming. In: Henning, G., Galli, M., Goneet, S. (eds.) XXXIV Conferência Latinoamericano de Informática, Santa Fe. Anales CLEI 2008, pp. 240–249 (2008)
23.
Simari, G., Parsons, S.: On the relationship between mdps and the bdi architecture. In: Proceedings of the Fifth International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS 2006, pp. 1041–1048. ACM, New York (2006)
24.
Simari, G., Parsons, S.: Markov Decision Processes and the Belief-Desire-Intention Model. Springer Briefs in Computer Science. Springer, Heidelberg (2011) CrossRefMATH
25.
Rens, G., Ferrein, A., Van der Poel, E.: A BDI agent architecture for a POMDP planner. In: Lakemeyer, G., Morgenstern, L., Williams, M.A. (eds.) Proceedings of the 9th International Symposium on Logical Formalizations of Commonsense Reasoning (Commonsense 2009), University of Technology, pp. 109–114. UTSe Press, Sydney (2009)
26.
Boutilier, C., Reiter, R., Soutchanski, M., Thrun, S.: Decision-theoretic, high-level agent programming in the situation calculus. In: Proceedings of the Seventeenth National Conference on Artificial Intelligence (AAAI 2000) and of the Twelfth Conference on Innovative Applications of Artificial Intelligence (IAAI 2000), pp. 355–362. AAAI Press, Menlo Park (2000)
27.
Chen, Y., Hong, J., Liu, W., Godo, L., Sierra, C., Loughlin, M.: Incorporating PGMs into a BDI architecture. In: Boella, G., Elkind, E., Savarimuthu, B.T.R., Dignum, F., Purvis, M.K. (eds.) PRIMA 2013. LNCS, vol. 8291, pp. 54–69. Springer, Heidelberg (2013) CrossRef
28.
Antos, D., Pfeffer, A.: Using emotions to enhance decision-making. In: Walsh, T. (ed.) Proceedings of the 22nd International Joint Conference on Artificial Intelligence (IJCAI 2011), pp. 24–30. AAAI Press, Menlo Park (2011)
29.
Murphy, R.: Introduction to AI Robotics. MIT Press, Massachusetts (2000)
30.
Roy, N., Gordon, G., Thrun, S.: Finding approximate POMDP solutions through belief compressions. J. Artif. Intell. Res. (JAIR) 23, 1–40 (2005)CrossRefMATH
31.
Paquet, S., Tobin, L., Chaib-draa, B.: Real-time decision making for large POMDPs. In: Kégl, B., Lee, H.-H. (eds.) Canadian AI 2005. LNCS (LNAI), vol. 3501, pp. 450–455. Springer, Heidelberg (2005) CrossRef
32.
Li, X., Cheung, W., Liu, J.: Towards solving large-scale POMDP problems via spatio-temporal belief state clustering. In: Proceedings of IJCAI-05 Workshop on Reasoning with Uncertainty in Robotics (RUR 2005) (2005)
33.
Shani, G., Brafman, R., Shimony, S.: Forward search value iteration for POMDPs. In: de Mantaras, R.L. (ed.) Proceedings of the 20th International Joint Conference on Artificial Intelligence (IJCAI 2007), pp. 2619–2624. AAAI Press, Menlo Park (2007)
34.
Cai, C., Liao, X., Carin, L.: Learning to explore and exploit in pomdps. In: NIPS, pp. 198–206 (2009)
35.
Shani, G., Pineau, J., Kaplow, R.: A survey of point-based pomdp solvers. Auton. Agent. Multi-Agent Syst. 27, 1–51 (2013)CrossRef
Metadata
Title
A Hybrid POMDP-BDI Agent Architecture with Online Stochastic Planning and Desires with Changing Intensity Levels
Authors
Gavin Rens
Thomas Meyer
Copyright Year
2015
Book
Agents and Artificial Intelligence

Print ISBN: 978-3-319-27946-6

Electronic ISBN: 978-3-319-27947-3

Copyright Year: 2015

DOI
https://doi.org/10.1007/978-3-319-27947-3_1

Premium Partner

    Image Credits