Tablet computer-supported conversation between people with dementia and their carers: technology as interactional focus

Data were collected within the project INdependent LIving support Functions for the Elderly, IN LIFE. IN LIFE was financed and carried out within the EU Research and Innovation programme Horizon 2020 [32]. The project held different work packages, and one focused on interaction of older people with dementia. This work package was run by the Swedish partners Dart—Centre for augmentative and alternative communication and assistive technology at Sahlgrenska University Hospital and CEDER—Center for dementia research at Linköping University. The sub-project constituting the interaction work package was vetted and received approval by the regional ethical review board (2015/162-31). Data analysis and the writing of this paper were done within the project “Life with dementia: communication, relations and cognition,” vetted and approved by the regional ethical review board (2017/469-31).

All participants were given information regarding the present study both verbally, presented in person by the research assistants, and through written text supported by pictures. The participants were ensured that they could withdraw their participation at any time without having to state any particular reason and that no consequences would follow from their withdrawal. The data collection was permitted by the participants through written consent.

Three dyads of women with dementia and paid carers participated in the study. The three women with dementia were over 65 years old and lived in care homes. Great care has been taken to ensure anonymity. No tests were used to assess speech, language and communication or cognition, and the researchers did not consult medical records. The women lived in homes that had accepted to participate in the study and were introduced to the research assistants by the staff working in the homes. The women have been given the assumed names Ada, Isa and Liv in the text. The three carers were younger and middle-aged women. Swedish was the first language of all participants. Using a tablet in interaction was a novel situation for the dyads.

Two readymade and earlier tested applications called CIRCA and CIRCUS were used in the study; that is, user testing and design were not purposes of the study. CIRCA [13, 23] is a web-based application developed from a previous version for stand-alone devices. CIRCA has access to a large database of generic pictures, videos and music files belonging to different categories that appear randomly on the tablet computer screen (Fig. 1). CIRCA was adapted to Swedish within the IN LIFE project, and the Swedish version was used in this study.

CIRCUS [33] has a personalized content with pictures and videos from a specific individual’s past and current everyday life (Fig. 2). For Ada and Isa, a mutual CIRCUS application was created comprising three albums with 48 pictures depicting common activities in the care home. In addition, Ada and Isa had one personal album each, containing personal pictures from their own photo albums, which were incorporated within the common CIRCUS application. For Liv, a personal application in CIRCUS was created comprising photos from her own photo albums.

Both applications work similarly to a digital photo album, where users can navigate between topics by touching photos or headings. In Fig. 1, by touching, for example, the heading “Famous people,” a collection of photos displaying famous people would appear on the screen. In Fig. 2, the sub-album “Family” has been chosen and the users now have the choice of looking at pictures from the Inca trail, birthday cakes or winter bathing.

Data included twenty-one recorded films of three dyads of persons with dementia and carers using tablet computers and the two different applications CIRCA (twelve recordings) and CIRCUS (nine recordings). The recordings were of varying length. The shortest recording was 6 min and 7 s, and the longest recording was 22 min and 48 s (Table 1).

The recording took place in the departments/rooms of each participant at times chosen by the participating carers and identified as the most appropriate for practical purposes. The carers got basic instructions on how to use the tablet and the two different applications, though no specific instructions were given regarding how to make conversation around the applications. As the main purpose of both CIRCA and CIRCUS is to support social conversation, there were no specific communication tasks assigned to the dyads. Instead, the dyads were instructed to use the applications in a way that suited their current interest and conversational needs. The dyads jointly chose from the pictures and topics available in the applications, even though the carers usually took on the main responsibility for handling the tablet computer.

First, the researchers looked at the twenty-one recordings together. Two main conversational topics were identified: talk relating to the pictures, films and music of the applications, and talk relating to the technology and the device. Second, the recordings were divided among the researchers and transcribed and analyzed individually using topical episode analysis [31, 34]. The purpose was to identify all sequences within the interactions in which the tablet and its functions per se were focused and forming unities, by speech or physical actions, by the dyads. A topical episode was defined as an interaction segment involving at least three consecutive contributions around the same main theme, that is, technology [35]. Topical episode analysis concerns how talk is made coherent through the participants’ drawing on different contextual resources such as background knowledge and artifacts and thus was considered relevant for the present analysis, albeit only instances of “talk about technology” were examined. The main focus for the current study is the various ways talk about technology becomes a conversational topic and in what situations, and therefore, an analytical method appropriate for both longer and shorter sequences of talk was needed. Moreover, the analysis needed to be able to take into account various contextual factors other than turns at talk. As the unit of analysis in topical episode analysis is a sequence (episode) rather than an utterance, and since the unit “episode” is of mid-range size (larger than a turn but smaller than a conversation), this method of analysis was suitable for the present data. Third, the researchers met and went through all the identified episodes of “technology talk” using transcripts and recordings to reach a consensus about which episodes to include in the data. The researchers agreed on most but not all episodes. The episodes which the researchers did not agree upon as clearly belonging to the category “technology talk” were omitted and not further treated. All episodes included in the final collection were hence approved by all researchers. Fourth, using transcripts and recordings the researchers collaborated in grouping the remaining episodes according to a more specific content. A preliminary coding resulted in twelve types of topical episodes. These, in turn, were restructured and condensed into six separate but not mutually exclusive types of episodes. The episodes represent the most common and typical ways of talking about technology in the recorded interactions. The episodes are presented along with their main characteristics and, in order to get a more detailed understanding of their place in the conversations, are analyzed sequentially following the basic features of interaction analysis [36].

Talk about tech problems reflects the fact that the tablet does not behave as expected. Engaging in conversation by means of using a tablet is a new activity for the dyads. Yet, they all have expectations regarding functionality, and when the tablet does not behave as expected, this is recurrently topicalized. The loading of content and the time it takes for selected pictures to occur on the screen, as well as for how long pictures are displayed on the screen, are common conversational topics. There are examples of pictures being presented both too slowly and too quickly (i.e. for a too short time). Slowness is more common though both variants exist and are featured in excerpt 1. Ada and the carer sit beside each other by the corner of a table. The tablet is in front of Ada and handled by the carer a bit from the side.

Ada and the carer have talked for almost 10 min and have had several problems with loading content when the carer tries to get some pictures to show. In line 4, the carer says: hmm it will probably show there soon, referring to the slowness and her selection of a picture in line 1. In lines 5 and 8, Ada responds by noticing that her sons were shown on the screen (line 5) and by commenting the speed of the presentation: though it went by so quickly (line 8). In line 10, the carer acknowledges Ada’s comment and again refers to the slowness: yes (1.7) no it was just as slow now I think.

Excerpt 1: Soon and quickly.

The second type of episode mainly concerns the dyads’ own handling of the tablet. The dyads explore the content and functions of the tablet and comment on their own handling of it. The comments in this category mainly refer to the functionality of the tablet and to the continuation of the activity. The dyads strive for the best possible function regarding, for example, the size of the pictures and thus adjust the tablet accordingly while, at the same time, orienting towards maintaining the activity. An example is given in excerpt 2 where Ada and the carer, sitting by the table, are looking at pictures of a social gathering.

Excerpt 2: Zooming.

The carer holds the tablet between them and Isa looks at it with her hands folded on her stomach.

Excerpt 3: Let’s see.

The carer selects different items on the screen, by tapping and dragging with her finger, and comments her own actions: now let’s see (1.3) what else we’ve got then you have to go backwards (lines 3–4) and and then you can (0.9) go backwards again (lines 6–7). Both these examples illustrate how the participants topicalize various aspects of using the tablet’s different functions (zooming, navigating, increasing sound volume, etc.), also in cases where the application works as expected.

The third episode type is labeled “expressing uncertainty in navigation” and reflects the fact that the dyads do not know how to handle the tablet. All the dyads display uncertainty regarding navigation of the application. Both carers and persons with dementia express uncertainty about how to handle the tablet and what to find “around the corner.” The dyads explore the tablet by selecting different pictures and these explorations are also reflected in their talk. Excerpt 4 originates from a conversation between Liv and her carer. The carer sits on the short side of the bed and Liv sits in a chair on the carer’s right side. The carer holds the tablet between the two of them and they talk and comment the pictures while trying to understand how the application is organized. The talk in lines 1 through 8 is typical for this conversation between Liv and her carer as they explore and navigate around in the application. Liv is reciting a psalm (line 1) and the carer comments this and a picture on the screen—the priest is singing marianne (line 2). She then tries to find something more to look at and talk about: now we’ll see if we can find anything more (.) nope (line 5).

Excerpt 4: Anything more?

The explorations are either successful, meaning the dyads find the picture, song or movie they are looking for, or anything else that is interesting enough to talk about, or, as is the case in excerpt 4, unsuccessful, which is demonstrated by the nope in line 5 and the now there were no more pictures there in line 8. Unsuccessful moves typically make the dyads try new ways, that is, tap other pictures.

The carers often explain how the tablet works and instruct both themselves and the persons with dementia in how to use it. Hence, there is a topical episode called “instructing and explaining.” Some explanations are introductions to the activity. These include descriptions of the application and the tablet as well as motives for using it. The aim of most instructions and explanations seems to be to get the person with dementia more involved. The carers explain how and what to do as well as possible consequences of these actions. In excerpt 5, the carer and Liv are sitting on the bed (carer) and in a chair (Liv). The carer holds the tablet between them. Liv has one hand on her knitting, and one hand on the chair. She looks at the tablet. The carer is very active as she both looks at and taps the screen while saying you press on them you just press (line 2) and like that I think then anything could appear (line 3).

Excerpt 5: You press!

In line 2, the first you is the second person pronoun du in Swedish, the second you is the indefinite Swedish pronoun man. This, and the insecurity expressed in lines 3 and 6, reflects the novelty of the situation and the fact that not only the person with dementia but also the carer and the dyad as such is in need of an increased understanding of how and what to do when using the tablet.

Sometimes unexpected things happen when the dyads are using the tablet, as if it was living a life of its own, and the dyads react to this by expressing surprise. The surprise is connected to the fact that the tablet seems to be interacting and actually responding to what they do with it, but also to the fact that it sometimes behaves in totally unpredictable ways. Unforeseen things just happen, as in excerpt 6.

Excerpt 6: Whoops.

The last type of topical episode reflects the dyads’ fascination by and interest in the development and state of technology and is labeled accordingly, “talk about technical development.” The dyads are amazed by the possibilities of the tablet. They express fascination by focusing the tablet itself, which is immediately present, but also take a wider perspective by, for example, making comparisons with the past and asking if and how “this thing” can be bought. In excerpt 7, Ada and the carer sit by the table with the tablet in front of Ada, the carer looks at it a little from the side.

Excerpt 7: Things of today.

They are looking at pictures from Ada’s childhood as Ada starts talking about the things that exist today (line 6: imagine the things that exist today (0.4) if one says) in comparison (line 7: like (0.3) if one would compare (0.9) with how it was) to before (line 8: when I was a child) when there were no such things as tablets (line 10: then these things didn’t exist). The carer responds (line 11) and the sequence is rounded up by Ada and the carer who notify and confirm the possibilities (lines 12–13).

Sometimes there is an element of insecurity, mixed with joy and enthusiasm, in how the persons with dementia approach and talk about the tablet. This is especially evident when they talk about how the pictures got into the tablet, rather than about what they show, as is the case in excerpt 8.

Excerpt 8: The entire photo book.

The data included twenty-one films of authentic tablet-supported interaction recorded in the homes of people with dementia. In this sense, the study is unique. However, only three dyads of older women with dementia and carers were involved and only two different applications were used. Hence, the findings are examples of digitally supported communication involving people with dementia but cannot be generalized to the larger population of people with dementia and carers or to other types of communication support. A detailed interaction and topical episode analysis was carried out, and assessment of inter-rater agreement was not a part of the methodology. Instead, individual and group analysis as well as consensus discussions were used. Episodes that the researchers did not agree upon were not included in the final analysis. The different ways of talking about technology that have been exemplified in this study are not mutually exclusive but rather overlap. Although the coding and analysis were performed with great detail, the decision of phenomena to highlight was in the hands of the analysts. There may very well be other ways in which technology is made relevant in the present data than those that have been presented in this study.

The findings of this study suggest that there are significant possibilities for technology in the daily lives of people with dementia and those who care for them. The present dyads can use new technology in terms of a tablet with a web-based application in order to support communication. They manage using such a device, show a sincere interest in its content and functions and seem to have enjoyable social conversations. The collaboration and shared exploration of the tablet as well as the problems the dyads face are relevant aspects of their interactions. An important finding with clear clinical implications concerns the amount of technical obstacles and the effects these have on language and conversation. Policy makers, directors and others in charge need to acknowledge that a tablet in itself is not enough. Successful implementation and use of technology depend on the technical prerequisites. Care homes for people with disability such as dementia need to be equipped so that the internet can be easily accessed and used.

Although user testing and design were not specific aims of the study, the findings from the present study have implications for future developmental work and design. The study acknowledges that the design of the application matters [24]. Two different types of applications were used in this study: a Swedish version of CIRCA [13, 23] including generic pictures, songs and films and CIRCUS [33] which was individually tailored with private pictures for each of the three women. The purpose was not to compare the two applications although we can conclude, from studying interaction, language and use of both that appearance and organization of content matters. Both CIRCA and CIRCUS are designed for people with cognitive difficulties associated with dementia and yet they are not self-evident and without hinder. The fact that there are no given start and end points but instead many different possibilities of ways to go and things to do, as is the case with CIRCUS in particular, seems to be inspiring and to stimulate interaction. However, as has also been shown, there are moments of considerable uncertainty where some kind of response from the system could have helped. For example, a clear signal telling the user that pictures are loading and how the loading process proceeds could have been useful. Also, from a social interaction point of view, multilayers of pictures, as is common in dynamically organized displays, are not efficient if most of the time is spent waiting for the pictures to show. Supposedly, several pictures of smaller size displayed at the same time would lead to easier and quicker access to different conversational topics for the dyads.

Reports focusing communication support in dementia are scarce. Future research should focus on different kinds of support for people with dementia and different conversational partners in a variety of daily environments and activities. Studies examining the effects of different designs and organization of dynamic displays in interactions involving people with dementia are welcome.