Performance of transesophageal echocardiography (TEE) requires two basic skills: the ability to obtain images and the ability to interpret these images. Since its inception in the 1980s, there has been a growing amount of educational resources aimed at TEE training and maintenance of competency. Educational materials that have been developed include textbook publications, journal articles, and courses. Most, if not all, of these approaches have targeted the skill of TEE image interpretation. The development of teaching resources aimed at developing the psychomotor skill of TEE image acquisition has been less successful to date. Similar to other traditional psychomotor training, TEE skills training has required a low instructor to student ratio, often 1:1. More importantly, TEE psychomotor training has most commonly been conducted with real patients in a real clinical context and, similar to other psychomotor clinical skills, typically requires repetition.1 There are limited clinical indications and opportunities for learning TEE, which has posed a significant barrier to the efficient teaching of this valuable skill. As such, this has limited the effective dissemination of the complete TEE skill set.2

Transesophageal echocardiography is not the first procedural skill requiring mastery of psychomotor elements. Surgical skills and endoscopy involve similar challenges to training.3 Simulation-based education that focuses on the training of psychomotor elements is becoming more widespread and is proving to have many benefits. Currently, the focus of work in surgical skill simulation is shifting to explore how best to optimize training and curriculum development.4 Today’s commercially available high-fidelity TEE simulation systems allow for the introduction of TEE simulation into the training of novice echocardiographers.2,5

The purpose of this study was to determine the effectiveness of an instructor-guided simulation curriculum in which a TEE simulator is used to train psychomotor skills and to compare it with self-directed online delivery of the same curriculum using the same TEE simulator.

Methods

The study was approved by The Ottawa Hospital Research Ethics Board, and written consent was obtained from all participants prior to the workshop. The study was conducted over four days in the spring of 2012.

After institutional review board approval, anesthesiologists with minimal or no TEE psychomotor experience were invited to participate in the study. Exclusion criteria included anesthesiologists who had any prior formal TEE training or instruction.

In addition to informed consent, all subjects completed a confidentiality agreement to ensure that test content was not disseminated between subjects. All subjects were given access to TEE learning materials four weeks prior to the study. These learning materials were accumulated by the study authors and made available on a password-protected Web site. Learning materials included introductory textbook chapters, journal articles, video clips, and Web-based interactive instructional tools. All of the available resources were thematically focused on TEE image acquisition. Learners were encouraged to perform a self-directed review of the available material. Access to the materials and time spent on review were not monitored.

Subjects were randomized to an instructor-guided hands-on TEE workshop (referred to as “Human” in all tables and figures) or to a self-guided electronic slide presentation (Microsoft Office PowerPoint 2007) TEE workshop (referred to as “Electronic” in all tables and figures). Both instruction groups used a commercially available high-fidelity TEE simulator for hands-on practice (CAE Healthcare VIMEDIX Simulator TEE module).

Prior to their instruction, all subjects completed a video-taped baseline pre-test of their ability to obtain TEE images using the simulator. Subjects were asked to obtain ten of the standard 20 TEE images required for a complete TEE examination as mandated by the National Board of Echocardiographers.6 The ten images chosen for the test have been previously described as the most important images to acquire7 (Fig. 1). Subjects were asked to obtain the best image possible for each of the ten images that would be assessed for quality. For each image, subjects were given a maximum of 45 sec to obtain their best view. Acquired final images were recorded for assessment. In addition to the psychomotor pre-test, all subjects were also given a ten-question written examination of their knowledge of TEE probe manipulation (Appendix 1; available as Electronic Supplementary Material).

Fig. 1
figure 1

Ten basic transesophageal echocardiography ultrasound images chosen for the pre- and post-tests

Following the pre-test, the instructor-guided group received a private structured and interactive lesson in psychomotor skills (4:1 instructor to student ratio) for two hours. Each subject had a dedicated 30 min of hands-on experience. All sessions were taught by the same instructor. Participants were instructed on how to manipulate and maneuver the TEE probe in order to obtain and optimize the previously discussed ten basic TEE images (Fig. 1). Instruction and feedback were primarily verbal; however, physical assistance with probe movement was also provided. The approach to instruction was consistent for all learners.

Participants in the self-guided instruction group were directed in their psychomotor skills lesson by an electronic slide presentation (Appendix 2; available as Electronic Supplementary Material). This step-by-step guide, created by the instructor of the instructor-guided group, described the required TEE probe manipulations using the four basic axes of TEE probe movement (advance/withdraw, rotate right/rotate left, anteroflex/posteroflex, angle plane). The content of the slide instruction was identical to the instructor-guided lesson. Using the written psychomotor instructions described on each slide and using the simulator as an adjunct, subjects obtained the ten basic TEE images. Subjects completed this lesson in groups of four over two hours. Sessions were designed so that each subject had a mandated dedicated 30 min of hands-on TEE experience. Groups guided themselves through the instructional presentation and relied on peer feedback to augment learning.

Following their respective training sessions, all subjects completed a video-taped post-test of both their TEE psychomotor ability and knowledge. Post-test images and written test questions were identical to pre-test (Fig. 1 and Appendix 1).

Two experienced National Board of Echocardiography certified TEE experts assessed all recorded final TEE images. Assessors were blinded to each other, to instructional modality, and to the phase of testing. Images were scored on a four-point Likert scale that was developed and validated through an iterative modified Delphi approach using four TEE experts not involved in the assessment of the images. Consensus on the scale was achieved after three rounds. Using the scoring rubric, a perfect exam score totalled 30 (Fig. 2).

Fig. 2
figure 2

Rubric used to score the candidates’ transesophageal echocardiography images

A single assessor, blinded to phase of testing and instructional modality, scored the ten-question written examination on knowledge of TEE probe manipulation.

Data analysis

Categorical variables were analyzed using Fisher’s exact tests and are presented as counts (proportions). Written examinations and TEE image acquisition scores were compared between the two groups using the non-parametric Mann-Whitney U test. The differences in scores between both the written examinations and TEE image acquisition scores (delta) were compared with a Wilcoxon Signed-Rank test. The inter-reader agreement between the expert assessors was performed using the Krippendorff α statistic. P values of less than 0.05 were considered to indicate statistical significance. All analyses were conducted using SAS® software, version 9.2 (SAS Institute, Cary, NC, USA).

Sample size was calculated based on our primary outcome for this repeated measures study design to show a change in performance from pre-test to post-test regardless of instructional method (G*Power software version 3.12, Düsseldorf, Germany).8 In the field of psychology and education, an effect size of > 0.8 standard deviation is considered large and acceptable for any given teaching intervention.9 Large effect sizes are used to determine effectiveness to justify the investment of time and resources for a teaching intervention. Assuming an effect size of 0.8 standard deviation and a power of 0.8, we calculated a minimum sample size of 15 subjects (α = .05 two-tailed) per instructional method arm of our study. Therefore, we needed to recruit and randomize a minimum of 30 subjects to the two different instructional method arms of our study (15 subjects per arm).

Results

Thirty-three subjects were included in this study. There were no differences in demographic distribution between the learner groups regarding sex, training level, and previous experience in TEE performance (Table).

Table Demographics

The median pre-training TEE simulator scores between the self-guided and instructor-guided groups were similar (9.0 vs 5.0, respectively; P = 0.28). Similarly, the median pre-training written examination scores between the two groups were similar (0.0 vs 1.0, respectively; P = 0.97).

Following instruction, the median post-training TEE simulator performance scores were significantly improved for both groups when compared with their pre-training TEE performance scores (P < 0.0001). The improvement scores (post-test scores minus pre-test scores) were not significantly different between the two instruction groups (12.5 vs 14.5; P = 0.55) (Figs. 3 and 4).

Fig. 3
figure 3

Change in transesophageal echocardiography score. This graph shows the average improvement (with confidence interval) in the two groups following instruction. Electronic refers to the electronic slide presentation instruction (Appendix 2; available as Electronic Supplementary Material). Human refers to human-guided instruction

Fig. 4
figure 4

Pre/post-test change in transesophageal echocardiography scores. Transesophageal echocardiography simulation scores in the two groups before and after instruction. Electronic refers to the electronic slide presentation instruction (Appendix 2; available as Electronic Supplementary Material). Human refers to human-guided instruction

Following instruction, the median post-training written examination scores were significantly improved for both groups when compared with their pre-training TEE performance scores (P < 0.0001). The median post-training improvement in written examination scores remained similar between the two groups (4.0 vs 6.0; P = 0.55) (Fig. 5).

Fig. 5
figure 5

Change in written test scores. This graph shows the average improvement (with confidence interval) in the two groups following instruction. Electronic refers to the electronic slide presentation instruction (Appendix 2; available as Electronic Supplementary Material). Human refers to human-guided instruction

There was very strong inter-rater reliability between the two examiners (α = 0.98; 95% confidence interval [CI]: 0.97 to 0.99).

Discussion

This study shows that a TEE curriculum incorporating a simulator as an adjunct significantly improves the ability of novice echocardiographers to perform a basic TEE examination in the short term. Interestingly, this improvement is observed regardless of instructional method. Specifically, learning took place whether guidance came from one-on-one training with an experienced echocardiographer or from a step-by-step written guide presented in an electronic slide presentation format. Following both methods of instruction, learners demonstrated the ability to maneuver the TEE probe correctly and to obtain interpretable images that form the basis of a TEE examination.

The role of intraoperative TEE is evolving. This procedure is no longer considered to reside solely in the domain of diagnosis, but it is also being recognized for its power as a monitoring device.10 Echocardiographic interrogation of both the left and right ventricles can provide a comprehensive assessment of ventricular function.11 Transesophageal echocardiography, when performed correctly, produces instantaneous images of cardiac anatomy in the context of real-time cardiac function. This combination of anatomy and physiology provides for a unique functional cardiovascular assessment and makes TEE an ideal cardiovascular monitoring tool.

The skill required to perform and interpret perioperative TEE is relatively complex and requires mastery of three distinct domains of knowledge: declarative, psychomotor, and intellectual. Declarative knowledge of TEE involves knowing the facts, figures, tables, and charts of TEE. It is the learning material obtained from textbooks, journal articles, and atlases.12 Psychomotor knowledge of TEE involves knowing how to move and position the TEE probe in order to produce an on-screen image. This knowledge involves a muscular component and a sequence of actions including “knobology”. Finally, intellectual knowledge involves applying data obtained from TEE images in concrete situations, analyzing the data to develop courses of action and to make judgments about their impact or value.

To date, there has been little development of curricula geared specifically to the psychomotor component of TEE.13 As with much psychomotor training, the skills training for TEE requires a ratio (often 1:1) of instruction on patients and requires much repetition.14 Combined with the limited clinical indications and opportunities, we consider acquisition of psychomotor skills to be the bottleneck and limitation to the dissemination of the complete TEE skill set.

Transesophageal echocardiography is not the first procedural skill to require the acquisition and mastery of psychomotor elements. Surgical skills and endoscopy involve similar challenges to training. With the advent and expansion of simulation, a revolution in surgical training has transpired in recent years.3 Simulation that focuses on psychomotor elements is becoming more widespread and proving to offer many benefits. In fact, development in psychomotor surgical skill simulation has now advanced beyond the stage of proof of concept, with simulation consistently being shown as beneficial.15 The focus of present work in surgical skill simulation has now shifted to the next level, i.e., optimizing training and curriculum.4 Efforts are now underway towards validating assessment tools for surgical and endoscopy skill mastery with emphasis on making the field more objective.16 At an even further extreme, and a means to show how far surgical skill simulation has advanced, use of simulation is now being considered as a prerequisite for admission to surgical specialties.17

Given the similarities, it is reasonable to assume that TEE psychomotor skills could be taught in a manner similar to surgery and endoscopy using simulation. Transesophageal echocardiography simulation has recently been introduced commercially and has been available for training at the University of Ottawa Skills and Simulation Centre since October 2010. In keeping with the rationale that intensive one- to two-day workshop training can be effective in teaching surgical skills, we developed and delivered a workshop to novice echocardiographers on basic TEE psychomotor skills.18,19 Staff anesthesiologists at this workshop were invited to participate in this study. The subjects of this study are representative TEE novices who are in clinical practice or beginning subspecialty training with access to a myriad of learning materials. Despite access to a wide variety of published learning content (a sample of the material was provided directly to our learners), our study showed that learners were unable to perform TEE at the onset of the workshop.12 The pre-test results clearly showed a lack of TEE psychomotor knowledge. Explanations for this include but are not limited to: inadequate learning materials, available resources were ineffective for teaching psychomotor skills, learners spent little (if any) time reviewing the educational materials prior to the instruction, or a local culture of pedagogical learning amongst learners, leading to little or no motivation for self-directed learning. Regardless of the etiology, it is our view that the construct of the study accurately represented the cohort of novice echocardiographers who are beginning TEE learning.

Following their instruction sessions, our cohort of learners clearly demonstrated that learning had occurred. Of particular interest in this repeated-measures study is the lack of difference between the method of instruction and the ability to acquire basic TEE psychomotor skills. Historically, medical training has followed Osler’s century-old approach of see one, do one, teach one.20 We have shown in this study that perhaps we have re-defined the requirements of the “see one” component. The ability of a learner to “see one” from an electronic slide presentation may have significant implications on the dissemination of this medical skill, as the presence of a human instructor is not required. The most labour-intensive and time-consuming elements, and thus primary sources for limiting dissemination of knowledge, are eliminated by substituting a digital instructional manual (electronic slide presentation) for a human instructor. We explain the success of digital instruction by applying one of Malcolm Knowles’ assumptions regarding the motivation of adult learning.21

As a person matures, his or her self-concept moves from one of a dependent personality toward one of a self-directed human being.

Learners went through the electronic slide presentation arm of the study in a self-directed manner. In that regard, the learning was learner-centred rather than instructor-centred. In addition, the electronic slide presentation allowed for group learning, discovery, and peer collaboration as there was no defined instructor expert. Though results of this study do not show superiority of this self-directed approach, its lack of inferiority and impact on self-directed learning makes it a very attractive approach to TEE psychomotor instruction.

There are several limitations to this study. First, we conducted a pre-test-post-test study without any significant time delay between assessments. The logistics of scheduling residents away from their clinical duties prevented us from optimizing the timing of the post-test; therefore, conclusions cannot be drawn regarding retention of this newly learned TEE skill. Second, this study was powered to detect our primary outcome, namely, a large and clinically significant effect of learning, and not a difference in learning between our modalities of instruction. Third, we lacked a control group to attempt the simulator in a similar time period without any instruction; therefore, we cannot confirm the learning effect of the simulator itself. Nevertheless, previous studies have shown that simulation in the absence of instruction does not confer learning.22

This study is also limited by the fact that a single instructor presented to all learners. We recognize that there is a normal variability in quality and effectiveness of instructors and their approach to instruction, which can affect learning. In addition, the electronic slide presentation was created by a single instructor and, similarly, its effectiveness could have differed depending on its quality. In both cases, however, there was relative uniformity in the instruction provided, and final results showed that a significant amount of learning did occur. In fact, the single instructor model can be viewed positively in that all learners received the same instruction, and therefore, no possible outcome differences should occur resulting from possible variations in instructional content due to different instructors. As a final point, despite the TEE simulator being considered high-fidelity, it remains a simulator, which calls into question whether the TEE skills learned on the simulator would translate to performance on a patient in clinical practice.

It the authors’ opinion, TEE will eventually become a mainstream hemodynamic monitor and intraoperative diagnostic tool in all operating rooms. Its growth and integration into mainstream anesthesia is still evolving, however, and despite all educational efforts made to date, dissemination of TEE knowledge and skills needs to be optimized.23 High-fidelity TEE simulators are an effective training adjunct for the acquisition of basic TEE psychomotor skills. There was no difference in improvement between the different deliveries of instruction (electronic slide presentation vs instructor). Further research will examine the ability to translate learning from simulator to real clinical scenarios and the need for a faculty resource for a curriculum in which a simulator is used as an adjunct.