User-centred design and evaluation of a tele-operated echocardiography robot

The World Health Organisation (WHO) regularly assesses the number of people working in the healthcare sector (doctors, nurses, other healthcare personnel) for all countries that publish numbers about employment in the healthcare sector. In the latest employment statistics, the WHO reports that in 2016 there was an estimated global needs-based shortage of health care workers of about 17.4 million workers [33]. The shortage of workers in the health care sector is also addressed in the WHO’s strategy for human resources for health which contains a global milestone for 2030 that “all countries are making progress towards halving inequalities in access to a health worker.” World Health Organization [30]. The usage of tele-operated robots for remote examinations is one way to guarantee better access to health specialists for people living in underdeveloped and rural areas. For example, Boman et al. [8] showed that the use of robot-assisted remote echocardiographic examination drastically reduces the time to diagnosis in rural areas. In a clinical trial that took place in the north of Sweden, they randomly assigned 19 patients to remote consultation and imaging, and 19 to standard consultation. The results of the study showed that the total process time from when the patient first contacted the doctor to when a correct diagnosis was issued was significantly reduced from 114 days to 26.5 days when using remote examination for consultation. The process time was reduced so drastically, because in a traditional consultation, patients first report to their primary healthcare centre and then, based on their condition, have to travel to secondary/tertiary centres for further examinations. When using remote examinations, patients only have to travel to their primary healthcare centre, but still can get access to additional expertise from doctors, who execute examinations remotely from other centres.

In the ReMeDi project (Remote Medical Diagnostician¹), we developed a robot system for remote echocardiography (Fig. 1). Shaw showed that the involvement of end users in the development is one necessary part for creating a successful medical product [35]. Thus, we followed a user-centred design (UCD) process for developing the robot. We involved the future end users of the robot (i.e., echocardiography specialists) in all development phases: requirement analysis, iterative system development, and system evaluation. In this paper, we report the collected findings of working together with end users to develop the ReMeDi robot. The main contributions of this work are: (1) Results of two eye tracking studies to compare traditional echocardiography examinations to remote examinations with the robot (Section 6); (2) results and best practices from the iterative development sessions with medical end users (Section 5); (3) a report on UCD methods to involve medical end users in requirement analysis (Section 4); and (4) an extensive discussion of the practicability and limitations of the UCD approach for developing a medical robot in an everyday clinical environment and the usability of eye tracking as comparative tool between traditional and remote examinations (Section 7). The main aims of this work is to provide new insights in using eye tracking as tool to compare task efficiency in traditional and tele-operated medical examinations and to share best practices when applying UCD methods in clinical practice.

Robots have been used for medical applications since the 1980s. The first medical robots were parts of computer-integrated surgery systems, e.g., [19]. Taylor and Stoianovici [40] give an overview for the first two decades of research on surgery robotics. Since the late 1990s, robots have also been used for ultrasound applications. Priester et al. [31] provide a review of medical robotic ultrasound systems. They define three classes of applications for ultrasound robots: extracorporeal imaging, needle-guidance, and intraoperative surgery. The ReMeDi system that we are describing in this paper (Section 3) is used for extracorporeal imaging, specifically for echocardiography. Other extracorporeal imaging systems have been proposed for ultrasonography (USG) of the abdomen (e.g., [15]), the carotid artery (e.g.,[32]), the lower limbs (e.g., [17]), and for general usage (e.g., [26]). Other groups have proposed systems for remote echocardiography, which are similar to the ReMeDi system. Boman et al. [9] presented the CARDISTA system (CARdiological consultation at a DISTance) and showed that long distance real-time echocardiography is technically possible. Koizumi et al. [18] presented a remote ultrasound diagnostic system for dialysis-related amyloid arthropathy and demonstrated in a diagnostic experiment a successful examination by a doctor on a real patient. Mathiassen et al. [27] showed that it is possible to realise tele-operated ultrasonography with a UR5 robot arm by Universal Robots. The robots of the other groups use similar technology to our system. However, the other groups do not report insights of working together with end users for system development.

We used eye tracking to measure the differences for doctors when conducting a traditional compared to a remote examination. Eye movements can provide insight into cognitive processes, such as attention and mental and visual workload. Eye tracking has been used in various fields, although not in the medical robotics context. The most commonly used eye tracking measurements are glances, fixations, and saccades. According to DIN EN ISO 15007-1 2013, a glance is defined as maintaining of visual gaze within an area of interest, bounded by the perimeter of the area of interest. A gaze may be comprised of more than one fixation and saccades to and from it. Its duration is measured as glance duration. A fixation is an alignment of the eyes so that the image of the fixated area of interest falls on the fovea (the middle of the retina responsible for our central, sharpest vision) for a given time period. A saccade is a brief, fast movement of the eyes that changes the point of fixation. In our eye tracking studies, we measured glances and fixations. Glances are used to measure the general attention of a person towards and area of interest. Fixations are used for visual information extraction, which is relevant for our study, because study from which areas of interest the doctors extract the most information. Saccades are informative for analysis of reading tasks or information search gaze patterns, which are both not relevant in our study.

Comparisons between traditional and remote examinations have been made before, although without using eye tracking. Arbeille et al. [2] asked doctors to execute a tele-operated ultrasound examination on 20 patients. As measurement for the reliability of the remote robotised system, they used the ratio between the number of well-visualised organs with the robotic system compared to the number of visualised organs in the traditional echography. The doctors were able to execute the examination correctly in 18 out of 20 cases, but needed approximately 50% longer than in a traditional examination. Smith-Guerin et al. [36] validated a tele-echography system on 20 patients. They found that post traumata could be correctly highlighted in 80% of the cases and four cardiac chambers views had been obtained by the doctor in 90% of the cases when using the remote system. The doctors were able to detect all digestive systems on liver, gall bladder, pancreas and kidney and all urinary symptoms on bladder, kidney, prostate, uterus and ovaries had been seen with remote and traditional examinations. Martinelli et al. [24] used a robot-based tele-echography system for remote ultrasound examination of the abdomen to examine 58 patients. They compared the diagnostic findings of the remote examination with that of a traditional examination. The doctors were able to find all aneurysms the patients had, using both examination techniques. Georgescu et al. [13] studied a doctor using a tele-operated echography robot for one year in everyday usage. The doctor used the robot 300 times over that year, examining abdominal organs, pelvic organs (6.7% of the examinations), supraaortic vessels (46%), the thyroid (11%), on leg veins (10%), and on the kidney and urinary tract (3.7%). These results show that tele-operated ultrasound is indeed a useful tool for doctors. Adams et al. [1] compared adult abdominal examinations using a tele-operated ultrasound system in a study with 18 patients. The doctors participating in this study identified 5 pathological findings in both examinations, but 3 findings were only identified in the traditional examination, and 2 findings were only identified using the tele-operated ultrasound robot. In all of the above reviewed comparison studies, the authors used either the number of correctly visualised organs or of found traumata as measurement for reliability of the remote examination. In the comparison study reported in this paper, we are focusing on echocardiography, i.e. ultrasonography of the heart. Our measurement criteria for a successful examination is the number of positions on the patient body, from which the doctors were able to correctly visualise the patient’s heart.

Finally, we review the application of user-centred design methods in medical robotics. User-centred Design (UCD) is a design methodology that involves the end user of a new product or human-machine interface in all product design stages: analysis, design and development, and evaluation. These stages are usually executed in an iterative, cyclical fashion. Figure 2 shows a general representation of the three stages. UCD has been recognised as product development methodology that leads to a better product usability and higher user acceptance [29]. It has also been formally standardised in ISO standard 9241-210:2019. In the development of the ReMeDi tele-operation system, we involved medical doctors and patients in focus groups and design sessions for analysis of user requirements; during the design phase of the robot we regularly organised feedback sessions with doctors to iterate robot prototypes; for evaluation of the system, we executed comparative eye tracking studies with medical doctors. Throughout this paper, we have used the key words analyis, design, and evaluation in section headers to make it clear to the reader which UCD method we describe in the section.

UCD has been used in the development of other medical technology. Martin et al. [23] summarise user-centred design methods and discuss their applicability for the development of medical devices. They recommend the use of contextual inquiry and ethnography at the beginning of development and other methods, including usability tests and focus groups during developing a new system. This is similar to our approach for user-centred design of the ReMeDi system. Shah et al. [14] show a survey of literature between 1980 and 2005 in which user-centred design methods have been used mostly for involving users in development and evaluation of healthcare technology. The analysis of the surveyed literature revealed that users are involved to different degrees in the development stages. Most often, they are involved in system design and in taking part in testing and trials. End users are less often involved in the deployment of new healthcare systems and very seldom take part in the concept stages of developing new systems. Similar to our approach, Brandt et al. [10] used a user-centred approach to collect the user requirements for an image-guided orthopaedic surgery robot. They executed a literature review, handed out questionnaires, and conducted workshops with domain experts. Similar to the second part of our requirement analysis, Lee at al. [20] conducted a survey with physiotherapists as part of a user-centred design process. Holt et al. [16] report on the involvement of therapists and users for a robotic system for rehabilitation therapy. They involved users not only in requirement analysis, but also in quarterly reviews to get feedback on system design and development. Lu et al. [22] present a user-centred design process for developing a limb stroke rehabilitation robot, which is similar to our approach. The authors observed stroke therapy sessions and conducted a survey with stroke therapists. They evaluated a first prototype of the robot with stroke therapists in a focus group and a preliminary study.

The work carried out and described in this paper has been part of the ReMeDi project. The goal of the project was to develop a robot that enables doctors to remotely execute echocardiography and auscultation on patients. Figure 1 shows the final implementation of the robot. It consists of two parts; one is stationed at the site of the doctor, the other is located at the site of the patient. The doctor controls the robot using a set of input devices (Fig. 1a), including a haptic interface for controlling the robot arm, a dedicated keyboard for the ultrasonic device, and a joystick to adjust the view angle of a camera on the robot to see the patient and robot arm.

The haptic interface was designed to let the doctors perform the same movements as during traditional examinations. The kinematics of the interface [5] cover the required workspace of the doctor and assure the stiffness of the overall device. It provides three degrees of freedom for the translation of the end effector and contains three legs supporting the end effector. Each leg is composed of two rotational joints, four bar linkages and a rotational joint. At the end of the interface, we installed a 3D-printed probe handle that is similar to that of standard ultrasonography probes used for echocardiography.

With a foot pedal (not visible in Fig. 1a), the doctor can switch between two robot arm movement modes. Doctors can either move the arm to position the ultrasonic probe or keep the tool centre point of the arm stable during the ultrasonography examination. The doctor further has three screens that show a live view from the patient site, the image from the ultrasonic device, and a three-way video conference system between doctor, assistant, and patient [39]. The robot is located at the patient site (Fig. 1b). It consists of a mobile platform [4], a screen that shows the doctor’s face to the patient, and a light-weight, compliant robot arm that can be equipped with either an ultrasonography probe or a stethoscope end effector. An assistant monitors the examination and ensures the safety of the patient. The two parts of the robot communicate over remote distances via the Internet.

For a general overview of the technical components of the ReMeDi robot please refer to [3]. A technical description of the control architecture is described in [37].

As first step in our user-centred design process, we conducted an extensive user requirement analysis. We began with a literature review in the fields of remote ultrasonography, physical examination, and robotic systems within a medical context. In the second step, we executed two observations of a health check performed by a doctor on a patient. The patient received two palpation examinations by the same doctor, the first one during a standard health check, the second one while being ill and having stomach pains. The observations allowed us to get a better feeling for the patients’ perspective during a physical examination and to generate a list of typically performed examination steps. In the third step, we organised two workshops with doctors, one in Austria and one in Poland. The main goals of the workshops were to get a deeper understanding for examination techniques, as well as to investigate when and why they are used. We also asked for the doctors’ opinions about the usage of robotics systems for remote medical examination. In the fourth step, we organised two focus groups with patients, again, one in Austria and one in Poland. The main goals of these workshops were to get a deeper insight into the patients’ perspective concerning examinations and about their opinions and needs regarding examinations executed remotely with a robotic system. Finally, we conducted two online surveys, one with doctors and another one with patients, in order to quantify the insights gained in the workshops.

The result of these analysis steps was a list of 21 user requirements (11 doctor, 3 assistant, 5 patient, 2 common requirements). In this publication, we focus on the implementation of the requirements by the doctors, hence we only discuss the 11 doctor requirements presented in Table 1. Please refer to [38] for more details of the user requirement analysis.

The first four requirements, RQ1–RQ4, cover aspects of communication and information exchange between doctors, assistants, and patients. In order to meet these requirements, we implemented a three-way video conference system [39] that has separate communication channels between doctor and patient (RQ1), and between doctor and assistant (RQ2). The video conference system also allows submission of additional patient data (RQ3, RQ4). Requirements RQ5 and RQ6 concern the control of the robot arm. For implementation of these requirements, the arm of the ReMeDi robot can be moved by hand, without using an additional steering device (RQ5). We iterated the placement of the camera facing the patient to ensure a visual observation of the patient by the doctor for the entire duration of the examination (RQ6). This is described in more detail in Section 5.2. There were two requirements about the ultrasonography examination, (RQ8, RQ9). We implemented RQ8 in the design of the probe handle used on the haptic interfaces as well as in the kinematics used to control the robot arm with the handle, as described in Section 3. We also implemented the three requirements for the other examination techniques percussion and palpation (RQ7, RQ10, RQ11), but do not discuss these here, since the focus of this paper is on ultrasonography.

We implemented the user requirements in the first prototype of the ReMeDi robot, which can be seen in Fig. 3. In the following section, we describe how we iterated the design of the robot together with the end users, which ultimately led to the development of the second prototype (Fig. 1) that we then used for the comparative user study described in Section 6.

In this section, we describe how we involved the end users in the design and development of the ReMeDi system. The goal here is to share our experiences of working together with users from the medical field and summarise the most important findings that are also relevant for development of other medical robots.

During the course of the project, a selected group of five medical doctors from the Medical University of Lublin, Poland, participated in designing and evaluating the ReMeDi system at different development stages. Virzi [42] showed that in a usability test 80% of the usability problems are detected with four or five participants. The doctors provided their feedback on an ongoing basis as soon as the engineers reached a new stage in the system design process. The group of doctors was involved in the following development steps:

In the following sections, we describe details of the evaluation for the first system prototype of the ReMeDi system. This evaluation stands as a good example of how we applied user-centred design methodology to involve the doctors in iterative system development. We review the deployed user-centred design methods in Section 5.1 and report the results of the evaluation in Section 5.2.

In this section, we describe the final step in the UCD cycle, the evaluation of the robot together with end users for which we used two eye tracking studies. The main goal of these studies was to compare focus of attention, task execution times, and gaze fixation of traditional echocardiography examination to remote examination with the ReMeDi system. For that, we asked doctors to wear a mobile eye tracker while performing the two types of examination. We specifically chose to use eye tracking studies to compare traditional to remote examination in order to see whether the gaze patterns and fixation times that user automatically apply in both applications are similar. We first describe the method, participants, and materials of both studies (Sections 6.1 and 6.2. After that, we report and compare the results of both studies (Section 6.3). Finally, we discuss the results in Section 7.

In this section, we first discuss the results of our comparison study and describe additional qualitative results to underpin our interpretation of the data. We then discuss the application of user-centred design methods in the medical context and our experience with applying these methods. Finally, we explain limitations of our work.

Our work shows that in medical robotics development the usage of user-centred design methods can lead to a product that meets the requirements of the doctors, who are the end users of the system. We furthermore presented that some of the methods have to be adjusted to the clinical context, especially usability tests, which have to be executed in a more flexible way and near the users. Eye tracking proved to be a useful tool for comparing traditional and remote examination, although there were differences in the structure of both examinations and we were not able to execute the remote examination with real patients.

Our results are relevant for researchers, who apply user-centred design methods in the medical robotics context and for medical robot system builders. Working together with end users is a somewhat time-consuming process, which has to be executed carefully. However, it leads to a more usable medical product. We also strongly believe that our work contributes to the WHO’s goal to increase access for patients in underdeveloped or sparsely populated areas to medical specialists.