Video connecting families and social robots: from ideas to practices putting technology to work

At the time of the outbreak of the COVID–19 pandemic, we were preparing a study that involved social robots at care homes. However, it quickly became clear that the pandemic had also affected the care context and especially the social circumstances there. We therefore decided to conduct a study on the experiences of the pandemic and its effects on technology usage and changing life and work practices. While no robots were present at this time, we found that other technology was lying around (to be described later in this section), and we were interested in investigating technology usage and attitudes towards future technologies, such as social robots, as a response to this crisis.

We collected our data in the two care homes at different time phases (see Fig. 1). Phase 1 was the strictest phase concerning the restrictions in Western Europe where our study took place, and it included most of the first lockdown (March–May 2020). In this phase, the care homes did not let other people enter, which also resulted in families being torn apart. We used this phase for method preparation and recruitment. Restrictions to receiving visitors were lifted rather quickly after the first lockdown and they were not reinstated with the same strictness later. In Phase 2 (May–June 2020), visits were allowed under severe restrictions, including window visits², or container visits³. A limited number of visitors per day were allowed, enhanced hygiene rules like wearing masks, and rapid COVID tests were in place. In Phase 3 (July 2020–Jan. 2021), visits were possible under lighter restrictions. We collected different types of data in Phase 2 and Phase 3 in the two care homes.

The two care homes that participated in the study are situated in a German-speaking country in Europe. The care homes are anonomyzed in this paper and they are referred to as H1 and H2. H2 is within a big city, while H1 is situated in a medium sized city. H2 is a long-term partner of the university of one of the co-authors and had a regular exchange with the researchers until the pandemic. Over a period of the last four years, the care home worked together and experimented with different digital tools like tablets, smart assistants and social robots. H1 has been also cooperating with the university on additional topics, like education.

Both care homes have religious roots and belong to catholic institutions. They are both connected to multiple other organisations within their buildings (H2: Kindergarten, Residential Assisted Living; H1: Monastery, Administration of Organisation). H2 is approximately twice the size of H1 in terms of permanent residents (119:49). H1 is connected to a monastery with nuns living within the care home.

In the study, we had a total of 18 participants: 9 residents and 9 care workers. The residents were all cognitively able. Before conducting the study, we acquired ethical approval from the university. The residents were invited by the care workers, and their participation was based on their willingness and ability to participate. The residents were between 55 and 93 years old, 4 of them male and 5 female. An overview of the participants is presented in Table 1.

We conducted a diary study and online interviews with 18 participants (see Table 1 and Fig. 1 for a timeline of methodology) in the two care homes⁴. We collected data from 9 residents by conducting a diary study and interviews (phone or video interviews), and from 9 care workers by conducting face to face interviews.

We created paper-based diaries for residents to fill out for four weeks per participant, with one entry per day. The diaries included open questions and playful activities (such as story completion and postcards (see e.g. Fig. 4) to be sent to researchers and a robot) to gather insights on the role of technology in day-to-day activities, on the experience of social contact and the pandemic, and on perspectives on the next generation of CT (i.e. robots), as we were interested in perspectives on the role of (both current and future) technologies triggered by the pandemic. The diaries were filled out by nine residents at two care homes (i.e. four and five, where the imbalance of numbers has been a result of the voluntary participation). To complement the long-term diary study, the participants who volunteered to fill out diaries were invited to participate in remote interviews with us using a chosen communication medium (e.g. telephone or video telephone using a videoconferencing software like Skype). Four interviews were conducted with residents of H2 via telephone, and five interviews were conducted with residents of H1 via video telephone, with the assistance of a care worker. To gather insights about care workers’ perspectives, we also conducted interviews with nine people working with residents, one from H2 and eight from H1 (based on the workers willingness to participate).

We recorded the interviews using recording software for audio/video. We transcribed the audios and videos and we coded them along with the diaries. The data was analysed with a reflexive thematic analysis approach [4] with the MaxQDA software by inductively clustering different topics. We generated, reviewed and defined themes to interpret results that we present below.

We collected data out of interest in the effects on the experiences of the pandemic on technology use, and therefore did not actively change the environment with research by asking participants to use technology (this was also not possible because of the restrictions, at least in the beginning of our study). However, both institutions had access to technical devices, even if in slightly different ways: Within the 6-years cooperation with the university of one of the co-authors, H2 had received around ten tablets and the residents were trained to use these devices by the university until the pandemic started. H1 on the other hand was not a long-term partner, however they received tablets from a national telecommunication company in Phase 1 and they did not receive any training.

As we also asked questions about attitudes towards robots (i.e. our intentional focus before the pandemic had started), we found mixed attitudes from residents. Some residents were sceptical about the potential for robots: “Modern technology can open new perspectives but can never replace human affection!” (R4). The resident sees technology as not smart: “Computers don’t think by themselves. Everything has to be inputted beforehand.” (R4). This opinion is not shared by everyone though as can be seen in a comment that was made about a robot from previous experiences at H2: “Yes, I like him. He is not too tall, he doesn’t frighten me and I want to have him with me” (R3). A resident who had experiences with Pepper wrote postcards to the robot, saying “I would love that you can pronounce my name [...]”, and “[...] tomorrow Pepper should come and visit me” (see Fig. 4). One resident who had previous experiences with Pepper even reported that his family was interested in the robot: “My kids want to get to know Pepper.” (R2).

The residents and care workers had many ideas regarding the functionalities of a robot and the tasks it could do for them. One care worker saw it as something that would be helpful to give orientation to people with dementia: “For the elderly, demented it could be something to give them orientation. [...] Maybe saying things like: ‘At noon we have this and this as lunch’ or ’This morning at 10:30 the Bingo game will happen’ (C2). The residents saw several tasks that a robot could do like cleaning or bringing things from one place to another. A resident who had previous experience with Pepper had a specific idea on how he would cooperate with a robot: “I am taller than him, I will work on the things that have to be done on the wall and he can help me on the ground.” (R3). A resident also thought the robot could measure the temperature of people (R2). One care worker however expected the robot to promote physical contact: “it is rather perhaps even contact-promoting or also motivates, rather to the participation in groups. And that could be counterproductive, of course, regarding Corona” (C1).

The pandemic has stimulated residents and care workers to deal with the impact of the pandemic by trying out new forms of social interaction and communication, especially with the use of CT like tablets. Care workers also experimented with video conference tools to conduct physical activities and support people. At some point, residents were also open to try out the use of tablets for information retrieval and to communicate with relatives.

However, digital communication practices have varied among older adults even long before the pandemic [3, 23], which is also reflected in our data showing only a small group of residents have actually used these new devices. Here, it would be possible to make these residents internal facilitators for other residents to help them with the use of tablets, as it has been the case in other research (albeit with active older adults not living in care homes) [39]. This could also be used to meet known barriers among older people such as asking for assistance with technology usage [14].

However, readiness to engage with technology that is lying around depended on various factors: Obviously, a crucial factor was the availability of tools lying around. Further, the willingness of care workers to support residents in digital communication was also crucial and required them to actively engage with technology. A lack of digital competence was also mentioned in one care home, tied to a lack of support by the government, social security and in care workers’ (ongoing) education. Care workers needed to adapt their work practice to the new forms of support they were giving to the residents and by doing so changed their work habits from helping family members to meet the residents within the care home to helping the residents and the families to use this CT. Readiness to engage with new technology could be promoted through the availability of devices, training and other support structures, institutional (work) culture, working conditions and the roles of workers. This has implications on relationships and the coordination of care networks, which connects to previous research in the care and healthcare context [17, 31, 50].

Crisis situations can change imaginaries of what is possible to respond to old and new problems. Digital transformation has been accelerated in various sectors during the pandemic [8, 13, 27, 29], and we see changes also in the care context to some extent. Most notable however, the pandemic experience has changed some imaginations of what is possible or what is valued, where solutions for old problems were found. Many relatives were already living far away from residents, resulting in very few visits from them and very little contact. The problem persisted. While technology had been there and relatives had been sometimes far away, the pandemic experience has triggered the usage of CT.

Engagement with new CTs and associated care practices requires an ecosystem perspective with interdependencies at least between politics, organizations, workers and residents. Care workers mentioned political agenda requirements to support the use of technology during work and to include aspects of digital competence into education and ongoing education of care workers. Institutions across care and politics can offer these opportunities and by doing so enhance digital literacy and access to state-of-the art technology. On an organizational level, the willingness of care workers and residents is needed to engage with technology in day-to-day work and life. The care institutions are also the ones who engage in institutional collaborations, for example, with research institutions and private corporations. Such corporations can provide access to participation in development, where participation can provide learning for older adults [26]. The care workers also need a certain level of readiness to engage with new tasks and use technology to fulfill their goals in day-to-day work, provided that they are offered opportunities by care institutions to do so, and that the technology is developed for them to actually support their daily work.

Last but not least, residents need a certain level of digital competence. This connects to previous COVID research which suggests a perspective to understand digital tools as they are being used by residents, not merely as an instrument but also as a learning process that needs professional support, infrastructure and training [16, 34].

The use of new CTs requires a collaborative perspective as they involved a triune of residents, care workers and family members. For the most part, digital devices were not used in a dyadic human-human interaction between a resident and a family member, nor has there been an omniscient and omnipotent agent [37] or user, but they have been used in a triple of residents, care workers and family members. The three parties are needed to establish a connection between each other. The family calls the care worker to make an appointment, the care worker then makes an appointment between the resident and the family. Once this is established, the communication still needs additional support. The care worker sets up the video call and quite often stays in the room with the residents to help out and to give an update to the family. This also means that all three parties have to appropriate a new technology and learn about the changing dynamics that come with it, where they need to articulate the distributed (work) activities [37] to interact and communicate in spite of distance. For the resident, it adds a layer of transparency, since in this situation they are present and talks between the family and the care worker about the resident and their general well-being are done with them instead of outside of their room. As shown by Lahtiranta et al. [24], care workers are building a bridge between the people who are not used to the technology and the technology itself. Furthermore, they are mediating the bridge to the family.

The use of CT adds to the workload which requires recognition, support structures and new work roles. The workload of care workers was increased during the pandemic also because of the three party arrangement. While one could argue that fewer visits may have freed up some time for the workers, health and care sectors may also rely heavily on adult family members’ support [29]. Also, care workers have mediated technology use on top of extra pandemic-related workload (e.g. due to changing hygiene concepts and evolving rules that came with the pandemic), which is related to the (mostly hidden) configuration work [2]. Care workers however saw a value in supporting residents to virtually communicate with family members and to use technology for sports and entertainment (even though one could argue that they were running the risk of extra workload in the future). This may have an impact on the organization of work and we see a potential for new work roles, as it is not clear yet who is going to formally take the work that technology brings in. Therefore, roles of care workers need to be re-defined, which may also involve changing job descriptions and specific time allocated. There is also a need for increasing staff and support structures to change work processes (which may also involve union involvement). In order to be sustainable, the work that technology brings in needs recognition and support. Furthermore, even if some older adults may be able to use technology independently after a while (as shown in previous case studies [45]), this may not be always the case given various barriers [14].

Before the pandemic started, we were in the middle of planning a study with a social robot to be deployed at the care homes. However, at the time of the outbreak in Europe, this has not been possible. It was the idea of engaging with off-the-shelf CT ‘lying around’ [15] that has been available to be picked up, not social robots.

While CT (i.e. tablets) that has been exploited during the pandemic may be much different from a robot, it is still worth discussing an outlook for future technologies and what can be learned from responses to this crisis for the next generation of CTs. This does not mean that the findings are directly transferable, especially as expectations towards robots are very specific and grounded in pre-conceptions [40]. However, it also became very clear in our study that certain aspects were not a matter of technology itself, but a matter of the context of living and working at care homes in how CT has been integrated into work and life.

First of all, readiness to engage with technology needs to be taken into account. Here, political and institutional requirements are important as well as the role of workers and residents that are not necessarily a matter of technology but a matter of the context. While robots are however not “lying around” usually, a limitation is here that robots may require a different set-up and even more training than already discussed in the case of these much simpler devices, i.e. tablets. The moderator role of care workers as part of a triune that we identified in the use of tablets may be even more important to consider for the use of robots (which also connects to previous work stating that the actions of care workers are crucial for facilitating human-robot interaction in certain care processes [6, 7, 21]). While robots are often envisioned in a dyadic human-robot constellation [21, 38], it is important to acknowledge the dependency of residents on care workers and their key role in configuration work. Our study has shown that CT usage can lead to new dependencies which requires future work to explore opportunities for new work roles or additional staff. While it is often discussed that robots offer more intuitive ways of interaction [1, 33, 47, 52], the reasons for this additional workload is also grounded in the task itself (rather than only interaction modalities of a technical artefact, which is nevertheless a key design aspect too). The work practices and ways of day-to-day care along with the crucial role of care workers for putting the communication with relatives to work must be certainly acknowledged also when designing and integrating future generations of CTs in this context, which also needs further exploration in future work.

On a methodological level, we aimed to adapt our research process continuously to the special needs and conditions of this time. This research also involved sensitive or difficult topics such as social isolation, experiences of being alone and related issues. Because methodology can affect people especially when they are being confronted with sensitive questions, we took a lot of time, research and preparation ahead to develop material (i.e. the diaries and the interview questions asked). The feedback has been mostly positive as people were able to reflect when filling out a diary, also dealing with emotions. As some care workers needed to assist people to fill out their diary (adding more work), this may have also facilitated a connection (and this very fact may have affected the data to a certain level). The postcards may have also shown how the residents connected with us and we got a better sense of their experiences.

Nevertheless, we were mostly constrained to use virtual data collection methods. While we explored different methods (e.g. diaries, story completion, postcards, and virtual interviews), there may be more suitable methods to gather insights of the field, such as in-situ interviews or ethnographic methods. The assistance by care workers in filling out diaries may have resulted in biases. Furthermore, we discussed the three party arrangement which involves residents, care workers and family members. We focused on the residents and the care workers as a key part of this arrangement. A limitation of our research is that we have not collected any data from the family members with this party as such, and which we propose to do in future work.

Last but not least, the pandemic came abruptly and affected life and work (not only at care homes), which has caused stress that has persisted. One of many other stress factors is the risk of infecting a highly vulnerable group. A limitation of this study may be therefore that the work practices may not be representative, as care workers may have also suffered from the effects of the pandemic in regard of metal and physical health.