Abstract
The quality experienced during the interaction with telephone-based spoken dialogue services results from a perception and judgement process. As a consequence, quality has to be measured in a subjective way, with the help of human test persons. To complement subjective quality judgements, parameters can be logged which quantify the flow of the interaction, the behaviour of the user and the system, and the performance of individual system modules during the interaction. Although such parameters are not directly linked to the quality perceived by the user, they provide useful information for system development, optimisation, and maintenance. This chapter presents standardised methods for both measurement approaches. Firstly, a brief overview of subjective evaluation experiments is provided, following Recommendation P.851 issued by the International Telecommunication Union. Secondly, a collection of parameters is presented which has proven to be useful for system design. An initial evaluation study in is described which shows that the parameters correlate only weakly with subjective judgements; thus, both types of evaluation provide complementary types of information. Linear regression models may be used to predict subjective judgements from interaction parameters, but their prediction accuracy is still limited.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2008 Springer Science + Business Media B.V.
About this chapter
Cite this chapter
Möller, S. (2008). Evaluating Interactions with Spoken Dialogue Telephone Services. In: Dybkjær, L., Minker, W. (eds) Recent Trends in Discourse and Dialogue. Text, Speech and Language Technology, vol 39. Springer, Dordrecht. https://doi.org/10.1007/978-1-4020-6821-8_4
Download citation
DOI: https://doi.org/10.1007/978-1-4020-6821-8_4
Publisher Name: Springer, Dordrecht
Print ISBN: 978-1-4020-6820-1
Online ISBN: 978-1-4020-6821-8
eBook Packages: Humanities, Social Sciences and LawSocial Sciences (R0)