A Case Study on the Relevance of the Competence Assumption for Implicature Calculation in Dialogue Systems

In order to enhance user acceptance, systems that generate utterances in dialogical scenarios as, e. g., question-answering systems or recommender systems with natural language interfaces, should show some degree of pragmatic behavior. In particular, dialogue systems face the challenge of generating utterances with sufficient scope for the user’s possible calculation of implicatures. In the following example, a virtual sales assistant S uses the scalar expression good, instead of the semantically stronger excellent.

S: The HP Laptop has a good AMD Radeon graphics coprocessor.

According to the standard view of quantity implicature calculation, a user, reading this statement, would reason as follows:

The system generated some \(\psi \) (good) instead of a semantically stronger alternative \(\phi \) (excellent). There must be reasons for not generating \(\phi \): Either the system does not believe \(\phi \) to be true or it believes \(\phi \) not to be true:

\(\lnot Bel_S(\phi )~\vee ~Bel_S(\lnot \phi )\)

Which interpretation holds, relies on the competence or experthood assumption: Only if it is shared knowledge of system and user that the system is competent in a way that allows them to predict the truth value of the semantically stronger \(\phi \), the user will eventually infer the stronger reading (Potts 2015). The inference can be outlined as follows:

Two different inferences are involved in this overall picture. The first inference concerns the first epistemic assignment (i. e., 2) that just states that the speaker does not know whether or not \(\phi \) holds. This so-called weak implicature was derived from the utterance \(\psi \) by following the Gricean quantity maxim. The strong or secondary implicature \(Bel_S(\lnot \phi )\) strengthens the weak implicature. It can be derived by considering the speaker’s competence: Either the speaker believes that \(\phi \) holds or that it doesn’t hold. For further details on the relation between implicatures and the competence assumption, see Geurts (2010) and Sauerland (2004).

The competence assumption thus is a crucial component of implicature calculation, because it prevents scalar expressions to be inferred with the stronger implicature reading by default. For the competence assumption to work, certain requirements must be met. Consider the following example:

S: I’ve heard a lecture at ComputerCon about that generation of graphics coprocessors. The HP Laptop has a good AMD Radeon graphics coprocessor.

In this example, the statement that the agent had heard a lecture on graphics coprocessors suggests that S is at least competent with respect to these processors. Hence, the user will infer without doubt that the graphics coprocessor in question is not excellent, which might affect his purchase decision more strongly than the weak implicature that the system just doesn’t know whether the coprocessor is excellent or not.

The aim of this paper is to explore this issue in the context of sales and recommender systems: We are interested in positive, neutral, and negative competence triggers and their influence on the competence assumption. Furthermore, we want to know whether the CA should be triggered locally (i. e., for single assertions or themes) or globally (i. e., for the overall dialogue).

Although there are a number of dialogue systems that deal with various aspects of implicature calculation, the role of the CA in legitimating implicature calculation has not been accounted for. Artificial agents have been utilized before, for either examining various pragmatic reasoning phenomena or maximizing dialogue efficiency, or both. For example, Vogel et al. (2013) use artificial agents to show that they behave in a Gricean manner to maximize their joint utility when faced with reference games or other interactional scenarios. The agent’s reasoning about the opponent’s belief states, modeled as a variant of the partially observable Markov decision process (POMDPs), to maximize joint utility, results in implicature-rich readings, but the weak–strong distinction has not been accounted for.

Stevens et al. (2016) show that sales dialogue efficiency can be enhanced with pragmatic question answering with indirect answers and consideration of user’s requirements by using a game-theoretic model of query answering for their agent: However, the implicature triggered by the indirect answer is, due to the probabilistic model of user types, a weak one only.

Schlöder and Fernández (2015) develop a model for pragmatic rejection by means of implicature(s). Efstathiou and Lemon (2014) consider an account on non-cooperative dialogue in automated conversational systems and teach their agents to behave in this manner. It was shown that in a trading game, non-cooperative behaviour such as deception could increase the agents performance in comparison to a cooperative agent. The CA does not play a role in these models as well.

Insights on the CA originate from linguistic and philosophical analyses (e. g., Geurts (2010)), but these works do not consider requirements for developing computational systems with a generation component in a dialogue setting.

Analogous to the situation in dialogue system research, the CA has not yet been a topic of empirical studies. The pragmatic approach in linguistics assumes that pragmatic reasoning is necessarily global (Sauerland 2004, p. 40), with which it refers to entire speech acts, not embedded sentences. In the context of this paper, global refers to an entire conversation, whereas local refers to a single speech act, primarily an assertion. It is not yet known on which level the CA is determined or whether it takes into account both levels of interpretation.

In our study, we confine ourselves to the question how the CA may be triggered and whether it will be established globally or locally. We consider the surface forms of the following aspects:

The results from this study on the role of the CA in dialogue systems suggest that the global competence assumption can be established fairly well. This holds especially true for the competence triggers of the category positive and negative, which had the most extreme competence ratings. In these categories, polite behaviour and indication of competence with mention of a certain position received the best mean ratings. The negative competence trigger with the lowest mean item number (8) worked with both attributes as well, whereas items number (7) and (9) – direct admission of incompetence and lack of time – didn’t compete. Future work thus should pay attention to the factor of polite behaviour and indication of competence. In terms of negative competence triggers, competence should be indicated in an indirect manner, as direct admission of incompetence did not score as well in the mean values.

Local competence triggers of postive-local and neutral-local, as well as neutral-global items failed to distinguish themselves. It should be considered whether neutral competence triggers are beneficial in the first place, or if they can more easily be computed by a lack of competence triggers. Furthermore, local competence triggers did not score as well as their global counterparts. Hence, for computing local competence a more blunt use of polite behaviours or competence indicators should be reviewed.

The global items of the categories positive and negative were the most promising and therefore suggest that it would be beneficial to integrate a model of CA into the information states of a dialogue system. These first results of our ongoing study on competence triggers will be used for subsequent studies on the role of the CA for deriving weak and strong implicatures. On the whole, they will provide the empirical grounding for a probabilistic model of content determination in a question-answering system in a sales scenario.