Towards Interactional Symbiosis: Epistemic Balance and Co-presence in a Quantified Self Experiment

Face-to-face conversation between humans implies a moment-by-moment organization of turns at talk, artifacts use and body postures that provide for the accountability of what is going on, what has been done, and what one could expect to be done. By ‘accountability’ is meant that social practices contain their very intelligibility as they occur in the here-and-now of activities. Analysts can rely on this situated intelligibility to describe and analyze how social activities are organized [17]. Social activity is structured as an emergent product of interrelations between sequential organization of talk, gestures, cognition and objects from the environment [22, 34, 43]. Therefore, social activity fait système and becomes a unit for analysts, because it is accountably treated as one by the social actors in the first place. Following Charles Goodwin’s definition of symbiosis [22], we will consider that through social interaction, humans are meant to organize some ‘wholes’ that are both different from, and greater than their parts, and constructed through the mutual interdependence of unlike elements.

There are many ways to describe the social ballet that is wound up each time people are co-present – and that would inspire research in Human-Machine Interaction. In this paper we present data collected from an experiment where epistemic balance was at stake. Essentially, in the frame of a musical quiz, we created situations where the robot would at some delimited occasions step into a person’s epistemic territory. The intended encroachment was triggered through a reflexive utterance produced by the robot during the interaction, that is, a turn-at-talk that would reflect traces of the person’s own activity [4]. The emergence of the reflexive turns relied on the participants’ physiological measurements that were measured with a connected Empatica E4 wristband [11, 16]. This epistemic encroachment was regularly responded to by the recipients. A close analysis of the following sequential environment ‘Reflexive Turn - Response’, will lead us to account for a collection of resources used by the participants in order to create a familiar social solidarity/synergy.

Leaning on human-human (or animal) interactions to support conversational agents or robots’ design, relies on a robust literature [5, 10, 13, 14, 28]. In the light of the development of social robots or socially interactive robots [13], concepts drawn from sociology such as connection, co-presence and cooperation, figure in definitions of engagement as ways to describe commitments or partnership in Human-Machine/Robot interactions [36] (HRI, HCI). However, such concepts are rarely scrutinized in their practical and sequential achievement. For the few researchers in HRI or HCI who have used it, Conversational analysis (CA), as a descriptive and naturalistic approach using interaction itself as a resource for analysis [45], provide analytical and methodological tools either to account for the moment-by-moment accomplishment of Human-Machine situated interactions [40, 41], either to design systems, and therefore anticipate on further interactions [37]. The present work is a contribution to this approach that aims at accounting for the sociality of robots as a practical accomplishment and therefore proposes an interactionist perspective on symbiosis. We assumed that giving access to the robot to something about the personal territory of an individual, like physiological data, could be questionable as something that helps, or somehow has an effect on the relation between this human and that robot.

The concepts of Quantified self and self monitoring are novel ideas to the field of HRI and we (the authors) are not aware of any experiments where this concept has ever been used in human-robot interaction. There are however several studies in the domain of human-computer interaction where the concept of quantified self has been employed. Li, Dey and Forlizzi [31] discuss how we can develop tools for analyzing the data that we collect about ourselves and how we can better perform self-reflection using this data. Human-computer interaction researchers have applied the concept of Quantified Self and self-tracking to diverse domains: to study electricity consumption [39], transportation habits [15], eating habits [46], and exercise habits [6]. While the use of smartphones and wearable devices continues to increase, we are finding new uses for these devices for self-monitoring such as measurement of our physical activity [12, 48], tracking our sleep patterns [1] and even looking at changes in our mood over time [50] among others which have become commercial successes. The E4 sensor and its predecessors have been used in a variety of experiments by researchers. Pieper and Laugero [38] used features collected using the Q sensor (predecessor to E4) in a study on preschool children and their emotional eating habits. Hernandez et al. [27] utilized the E4 to study stress during driving. In our experiment, we utilized only three of these measures: galvanic skin response [3, 7], pulse rate and peripheral skin temperature and two derived measures: slope of the galvanic skin response and change in pulse rate.

Largely, we observed that reflexive turns had recurrently an effect on participants interacting with the robot. That is to say, when the robot claims, one way or another, an access to the participant’s personal territory, the latter displays some reaction to it. We intend to show that those reactions account for socio-organic solidarities as proofs of interactional symbiosis. First, there are behaviors that are general to social interactions, namely co-presence management issues and preference for agreement. Second, there are more specific practices related to the epistemic encroachment that participants undergo in the experiment.

In this experiment we used a quantified-self device to provide the robot with the presupposition of a specific epistemic authority vis-à-vis the participant, and we tested this authority through reflexive sequences. Largely, we found (1) that reflexivity taken care of by the robot, has an effect on the participants’ behavior, as a step into their personal epistemic territory, and (2) that the persons display practices that show the analogous commitment as in human-human interactions regarding the preferential organization of turns-at-talk in terms of adjacency, agreement, and epistemic balance. Even if they are aware of the robot’s limitations, participants display an attention to organize a participation framework (with rights and obligations), in which the robot is treated like a participant in its own right. Organization is what binds elements, events or individuals in a symbiotic relationship, that is, a potential synergy in which different sign systems work together to build relevant action and accomplish consequential meaning. Therefore, the scope of turn-design must not be limited to stream of speech phenomena (grammar, speech acts), but must encompass structures providing for the organization of the endogenous activity systems within which strips of talk are embedded [23]: 370.

As Foster [14] put it, detailed analysis of interactional behaviors offers opportunities for socially interactive robots design improvements, that is: identify and reproduce human ordinary skills (perception, practical reasoning, gesture…) in order to make the machines more adaptable regarding interactional situations (assistive robots in the home environment, companion robot). We observed in the data that the participants treat the robot as a body, a presence. It shows that social interaction, as Goffman identified it decades ago, is first, before talking, a matter of hand-to-hand management: entering in a mutual perceptive field, focusing on a common object – depending on the participation footing of the participant. Moreover, we introduced the problem of epistemic balance, and observed some occurrences of plasticity of the epistemic configurations. Epistemic balance phenomena analysis points out that social synergy is a dynamic process: status in the interaction are to be defined and may be negotiated. Epistemic configurations play also a fundamental role in the way higher-order classes of action such as suggestions, proposals, or offers are dealt with in the interaction. We believe this is an entire issue to explore in HRI.

Hence, we observed all sorts of practices that are grist to the interactional mill and accountable arguments for a view of humans working at establishing and maintaining socio-organic solidarities with a robot. This was done in a short term and a quite artificial setting. Prospectively, we would need a larger amount of data to extend the analysis towards (a) more natural interactions, (b) more differentiation in reflexive turn-design impact, (c) better understandings of the usefulness of physiological measurements as resources for Human-Machine situated symbiosis.