nach oben

Erschienen in:

2016 | OriginalPaper | Buchkapitel

Decomposing Drama Management in Educational Interactive Narrative: A Modular Reinforcement Learning Approach

verfasst von : Pengcheng Wang, Jonathan Rowe, Bradford Mott, James Lester

Erschienen in: Interactive Storytelling

Verlag: Springer International Publishing

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

Recent years have seen growing interest in data-driven approaches to personalized interactive narrative generation and drama management. Reinforcement learning (RL) shows particular promise for training policies to dynamically shape interactive narratives based on corpora of player-interaction data. An important open question is how to design reinforcement learning-based drama managers in order to make effective use of player interaction data, which is often expensive to gather and sparse relative to the vast state and action spaces required by drama management. We investigate an offline optimization framework for training modular reinforcement learning-based drama managers in an educational interactive narrative, Crystal Island. We leverage importance sampling to evaluate drama manager policies derived from different decompositional representations of the interactive narrative. Empirical results show significant improvements in drama manager quality from adopting an optimized modular RL decomposition compared to competing representations.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Vorheriges Kapitel Delayed Roles with Authorable Continuity in Plan-Based Interactive Storytelling

Nächstes Kapitel Bringing Authoritative Models to Computational Drama (Encoding Knebel’s Action Analysis)

There are 13 AESs in Crystal Island. In this work, we focus on four AESs, which were chosen because they were the most commonly occurring in our training corpus.

Rowe, J., Lester, J.: Improving student problem solving in narrative-centered learning environments: a modular reinforcement learning framework. In: Conati, C., Heffernan, N., Mitrovic, A., Verdejo, M.F. (eds.) 17th International Conference on Artificial Intelligence in Education. LNCS, vol. 9112, pp. 419–428. Springer, Heidelberg (2015)CrossRef

Lee, S., Rowe, J., Mott, B., Lester, J.: A supervised learning framework for modeling director agent strategies in educational interactive narrative. IEEE Trans. Comput. Intell. AI Games 6, 203–215 (2014)CrossRef

Li, B., Lee-Urban, S., Johnston, G., Riedl, M.: Story generation with crowdsourced plot graphs. In: 27th AAAI Conference on Artificial Intelligence, pp. 598–604 (2013)

Yu, H., Riedl, M.: Personalized interactive narratives via sequential recommendation of plot points. IEEE Trans. Comput. Intell. AI Games 6(2), 174–187 (2014)CrossRef

Orkin, J., Roy, D.: Understanding speech in interactive narratives with crowd sourced data. In: 8th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, pp. 57–62 (2012)

Nelson, M., Roberts, D., Isbell, Jr., C., Mateas, M.: Reinforcement learning for declarative optimization-based drama management. In: 5th International Conference on Autonomous Agents and Multi-Agent Systems, pp. 775–782 (2006)

Roberts, D., Nelson, M., Isbell, C., Mateas, M., Littman, M.: Targeting specific distributions of trajectories in MDPs. In: 21st AAAI Conference on Artificial Intelligence, pp. 1213–1218 (2006)

Thue, D., Bulitko, V.: Procedural game adaptation: framing experience management as changing an MDP. In: Fifth Workshop on Intelligent Narrative Technologies, pp. 44–50 (2012)

Rowe, J., Mott, B., Lester, J.: Optimizing player experience in interactive narrative planning: a modular reinforcement learning approach. In: 10th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, pp. 160–166. AAAI Press, Menlo Park, CA (2014)

10.

Sutton, R., Barto, A.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)MATH

11.

Peshkin, L., Shelton, C.: Learning from scarce experience. In: Proceedings of the Nineteenth International Conference on Machine Learning, pp. 498–505 (2002)

12.

Riedl, M., Bulitko, V.: Interactive narrative: an intelligent systems approach. AI Mag. 34(1), 67–77 (2012)

13.

McCoy, J. Treanor, M., Samuel, B., Mateas, M., Wardrip-Fruin, N.: Prom week: social physics as gameplay. In: 6th International Conference on Foundations of Digital Games, pp. 319–321 (2011)

14.

Suttie, N., Louchart, S., Aylett, R., Lim, T.: Theoretical considerations towards authoring emergent narrative. In: Koenitz, H., Sezen, T.I., Ferri, G., Haahr, M., Sezen, D., Catak, G. (eds.) ICIDS 2013. LNCS, vol. 8230, pp. 205–216. Springer, Heidelberg (2013)CrossRef

15.

Porteous, J., Lindsay, A., Read, J., Truran, M., Cavazza. M.: Automated extension of narrative planning domains with antonymic operators. In: 14th International Conference on Autonomous Agents and Multiagent Systems, pp. 1547–1555 (2015)

16.

Robertson, J., Young, R.: Automated gameplay generation from declarative world representations. In: 11th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, pp. 72–78 (2015)

17.

Lamstein, A., Mateas, M.: Search-based drama management. In: 2004 AAAI Workshop on Challenges in Game Artificial Intelligence, pp. 103–107 (2004)

18.

Singh, S.: Transfer of learning by composing solutions of elemental sequential tasks. Mach. Learn. 8, 323–339 (1992)MATH

19.

Dietterich, T.: Hierarchical reinforcement learning with the MAXQ value function decomposition. J. Artif. Intell. Res. 13, 227–303 (2000)MathSciNetCrossRef

20.

Chi, M., VanLehn, K., Litman, D., Jordan, P.: Empirically evaluating the application of reinforcement learning to the induction of effective and adaptive pedagogical strategies. User Model. User-Adap. Inter. 21, 137–180 (2011)CrossRef

21.

Mandel, T., Liu, Y., Levine, S., Brunskill, E., Popovic, Z.: Offline policy evaluation across representations with applications to educational games. In: 13th International Conference on Autonomous Agents and Multi-Agent Systems, pp. 1077–1084 (2014)

22.

Riezler, S., Maxwell, J.: On some pitfalls in automatic evaluation and significance testing for MT. In: Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization, pp. 57–64 (2005)

Titel: Decomposing Drama Management in Educational Interactive Narrative: A Modular Reinforcement Learning Approach
verfasst von: Pengcheng Wang
Jonathan Rowe
Bradford Mott
James Lester
Verlag: Springer International Publishing
Buch: Interactive Storytelling
Print ISBN: 978-3-319-48278-1

Electronic ISBN: 978-3-319-48279-8

Copyright-Jahr: 2016
DOI: https://doi.org/10.1007/978-3-319-48279-8_24

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"