The GenAI4IDN Benchmark 2.0 - AI-Assisted Assessment of Generative AI Tools for the Design of Interactive Digital Narratives

The GenAI4IDN Benchmark 2.0 chapter provides an in-depth evaluation of the latest generative AI tools for interactive digital narrative (IDN) design. The benchmarking framework has been updated to include 15 tools, covering a wider range of GenAI models, including open-source offerings. The chapter introduces two new aspects to the framework: iterative prompting and the impact of contextualized prompting. It also devises an AI-assisted evaluation method to handle the sheer volume of outputs generated during the benchmarking process. The evaluation covers 15 common IDN design tasks, each with four sub-tasks, and includes a detailed analysis of the tools' capabilities in text, code, image, video, and music generation. The results show significant improvements in image generation, code generation, dialogue trees, and background stories. The chapter concludes with a discussion on the limitations of current GenAI tools and the ethical concerns surrounding their use. The findings highlight the potential of GenAI tools in IDN design, but also emphasize the need for proper planning, training, and ethical considerations. This chapter is a valuable resource for professionals looking to understand the current state of GenAI tools and their applications in interactive digital narrative design.