Skip to main content

Advertisement

SpringerLink
Log in

Construct validity of the ProMIS laparoscopic simulator

  • Published:
Surgical Endoscopy Aims and scope Submit manuscript

Abstract

Background

Laparoscopic surgery challenges both the surgical novice and experienced open surgeon with unique psychomotor adaptations. Surgical skills assessment has historically relied on subjective opinion and case experience. Objective performance metrics have stimulated much interest in surgical education over the last decade and proficiency-based simulation has been proposed as a paradigm shift in surgical skills training. New assessment tools must be subjected to scientific validation. This study examined the construct validity of a hybrid laparoscopic simulator with in-built motion tracking technology.

Methods

Volunteers were recruited from four experience groups (consultant surgeon, senior trainee, junior trainee, medical student). All subjects completed questionnaires and three tasks on the ProMIS laparoscopic simulator (laparoscope orientation, object positioning, sharp dissection). Motion analysis data was obtained via optical tracking of instrument movements. Objective metrics included time, path length (economy of movement), smoothness (controlled handling) and observer-recorded penalty scores.

Results

One hundred and sixty subjects completed at least one of the three tasks. Significant group differences were confirmed for number of years qualified, age and case experience. Significant differences were found between experts and novices in all three tasks. Sharp dissection was the strongest discriminator of four recognised laparoscopic skill groups: consultants outperformed students and juniors in all three performance metrics and objective penalty score (p < 0.05), and only accuracy of dissection did not distinguish them from senior trainees (p = 0.261). Seniors dissected faster, more efficiently and more accurately than juniors and students (p < 0.05).

Conclusions

ProMIS provides a construct valid laparoscopic simulator and is a feasible tool to assess skills in a cross-section of surgical experience groups. ProMIS has the potential to objectively measure pre-theatre dexterity practice until an agreed proficiency level of dexterity is achieved. Future work should now examine whether training to expert criterion levels on ProMIS correlates with actual operative performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Carlin AM, Gasevic E, Shepard AD (2007) Effect of the 80-hour work week on resident operative experience in general surgery. Am J Surg 193:326–330

    Article  PubMed  Google Scholar 

  2. Chikwe J, de Souza A (2004) Pepper JR (2004) No time to train surgeons. BMJ 328:418–419

    Article  PubMed  Google Scholar 

  3. Heath RM, Gate T, Halloran C, Callaghan M, Paraoan M, Blair SD (2005) The European Working Time Directive ‘triple whammy’: hitting surgical trainees where it hurts. Br J Surg 92(Supp 1):113

    Google Scholar 

  4. Modernising Medical Careers: the response of the four UK Health Ministers to the consultation on Unfinished Business: Proposals for Reform of the Senior House Officer Grade. 26-2-0003

  5. UK Health Departments. Modernising Medical Careers: the next steps. 15-4-2004. London, Department of Health

  6. http://www.iscp.ac.uk. 2006

  7. Paisley AM, Baldwin PJ, Patterson-Brown S (2001) Validity of surgical simulation for the assessment of operative skill. Br J Surg 88:1525–1532

    Article  PubMed  CAS  Google Scholar 

  8. Darzi A, Mackay S (2001) Assessment of surgical competence. Qual Health Care 10:ii64–ii69

    Article  PubMed  Google Scholar 

  9. Figert P, Park A, Witkze D, Schwartz R (2001) Transfer of training in acquiring laparoscopic skills. J Am Coll Surg 193:534–537

    Article  Google Scholar 

  10. Satava RM (1993) Virtual reality surgical simulator: the first steps. Surg Endosc 7:203–205

    Article  PubMed  CAS  Google Scholar 

  11. Gallagher AG, Ritter EM, Champion HR, Higgins G, Fried GM, Moses G, Smith CD, Satava RM (2005) Virtual reality simulation for the operating room: proficiency-based training as a paradigm shift in surgical skills training. Ann Surg 241:364–372

    Article  PubMed  Google Scholar 

  12. Gallagher AG, Ritter EM, Satava RM (2003) Fundamental principles of validation and reliability: rigorous science for the assessment of surgical education and training. Surg Endosc 17:1525–1529

    Article  PubMed  CAS  Google Scholar 

  13. Streiner D, Norman G (1989) Health measurement scales. New York, Oxford University Press

    Google Scholar 

  14. Derossis AM, Fried GM, Abrahamowicz MA, Sigman HH, Barkun JS, Meakins JL (1998) Development of a model for training and evaluation of laparoscopic skills. Am J Surg 175:482–487

    Article  PubMed  CAS  Google Scholar 

  15. Gallagher AG, Lederman AB, McGlade K, Satava RM, Smith CD (2004) Discriminitive validity of the Minimally Invasive Surgical Trainer in Virtual Reality (MIST VR) using criteria levels based on expert performance. Surg Endosc 18:660–665

    Article  PubMed  CAS  Google Scholar 

  16. Schijven MP, Jakimowicz J (2003) Construct validity: experts and novices performing on the Xitact LS500 laparoscopy simulator. Surg Endosc 17:803–810

    Article  PubMed  CAS  Google Scholar 

  17. Royal College of Surgeons of England (2006) Inquiry into independent sector treatment centres. Ann R Coll Surg 88(Suppl):116–119

    Google Scholar 

  18. Cruelty to Animals Act. 39 and 40 Vict, 1–8. 1876. Ch77

  19. Mahadevan V (2006) The Human Tissue Act 2004: implications for anatomical work at the college. Bulletin of the Royal College of Surgeons of England 88:264–265

    Article  Google Scholar 

  20. Reznick R, MacRae H (2006) Teaching surgical skills—changes in the wind. N Engl J Med 355:2664–2669

    Article  PubMed  CAS  Google Scholar 

  21. Gaba D (2004) The future vision of simulation in healthcare. Qual Safety Healthcare 13:2–10

    Article  Google Scholar 

  22. Kneebone R (2003) Simulation in surgical training: educational issues and practical implications. Med Educ 37:267–277

    Article  PubMed  Google Scholar 

  23. Scott D, Bergen PC, Rege RV, Laycock R, Tesfay ST, Valentine RJ, Euhus D, Jeyarajah R, Thompson W (2000) Jones DB (2000) Laparoscopic training on bench models: better and more cost effective than operating room experience. J Am Coll Surg 191:272–283

    Article  PubMed  CAS  Google Scholar 

  24. Rosser JC, Rosser LE, Savalgi RS (1997) Skill acquisition and assessment for laparoscopic surgery. Arch Surg 132:200–204

    PubMed  CAS  Google Scholar 

  25. Smith CD, Tung P (1998) Assessing laparoscopic manipulative skill: beyond the stopwatch. Surg Endosc 12:492

    Google Scholar 

  26. Schmidt R, Wrisberg C (2000) Motor Learning and Performance. Champaign, IL, Human Kinetics

    Google Scholar 

  27. Moynihan BGA (1920) Ritual of a surgical operation. Br J Surg 8:27–35

    Article  Google Scholar 

  28. Adrales G, Donnelly M, Chu U, Witkze D, Hoskins J, Mastrangelo M Jr, Gandsas A, Park A (2004) Determinants of competency judgments by experienced laparoscopic surgeons. Surg Endosc 18:323–327

    Article  PubMed  CAS  Google Scholar 

  29. Martin J, Regehr G, Reznick R, MacRae H, Murnaghan J, Hutchison C, Brown M (1997) Objective structured assessment of technical skill (OSATS) for surgical residents. Br J Surg 84:273–278

    Article  PubMed  CAS  Google Scholar 

  30. Moorthy K, Munz Y, Dosis A, Bello F, Chang A, Darzi A (2004) Bimodal assessment of laparoscopic suturing skills: construct and concurrent validity. Surg Endosc 18:1608–1612

    PubMed  CAS  Google Scholar 

  31. Broe D, Ridgway P, Johnson S, Tierney S, Conlon C (2006) Construct validation of a novel hybrid surgical simulator. Surg Endosc 20:900–904

    Article  PubMed  CAS  Google Scholar 

  32. Gallagher AG, Satava RM (2002) Virtual reality as a metric for the assessment of laparoscopic psychomotor skills. Surg Endosc 16:1746–1752

    Article  PubMed  CAS  Google Scholar 

  33. Gallagher AG, Smith CD, Bowers SP, Seymour NE, Pearson A, McNatt S, Hananel D, Satava RM (2003) Psychomotor skills assessment in practicing surgeons experienced in performing advanced laparoscopic procedures. J Am Coll Surg 197:479–488

    Article  PubMed  Google Scholar 

  34. Grantcharov TP, Bardram L, Funch-Jensen P, Rosenberg J (2003) Learning curves and impact of previous operative experience on performance on a virtual reality simulator to test laparoscopic surgical skills. Am J Surg 185:146–149

    Article  PubMed  Google Scholar 

  35. Grantcharov TP, Rosenberg J, Pahle E, Funch-Jensen P (2001) Virtual reality computer simulation: an objective method for the evaluation of laparoscopic surgical skills. Surg Endosc 15:242–244

    Article  PubMed  CAS  Google Scholar 

  36. Grantcharov TP, Kristiansen VB, Bendix J, Bardram L, Rosenberg J, Funch-Jensen P (2004) Randomized clinical trial of virtual reality simulation for laparoscopic skills training. Br J Surg 91:146–150

    Article  PubMed  CAS  Google Scholar 

  37. Pearson A, Gallagher AG, Rosser JC, Satava RM (2002) Evaluation of structured and quantitative training methods for teaching intracorporeal knot tying. Surg Endosc 16:130–137

    Article  PubMed  CAS  Google Scholar 

  38. Seymour NE, Gallagher AG, Roman SA, O’Brien MK, Bansal VK, Anderson DK, Satava RM (2002) Virtual reality training improves operating room performance: results of a randomized, double-blinded study. Ann Surg 236:458–464

    Article  PubMed  Google Scholar 

  39. Chaudhry A, Sutton C, Wood J, Stone R, McCloy R (1999) Learning rate for laparoscopic surgical skills on MIST VR, a virtual reality simulator: quality of human-computer interface. Ann R Coll Surg 81:281–286

    CAS  Google Scholar 

  40. Schijven MP, Klaassen R, Jakimowicz J, Terpstra OT (2003) The intercollegiate basic surgical skills course. Surg Endosc 17:1978–1984

    Article  PubMed  CAS  Google Scholar 

  41. Woodrum D, Andreatta P, Yellamanchilla R, Feryus L, Gauger P, Minter R (2006) Construct validity of the LapSim laparoscopic surgical simulator. Am J Surg 191:28–32

    Article  PubMed  Google Scholar 

  42. McNatt S, Smith CD (2001) A computer-based laparoscopic skills assessment device differentiates experienced from novice laparoscopic surgeons. Surg Endosc 15:1085–1089

    Article  PubMed  CAS  Google Scholar 

  43. Datta V, Chang A, Mackay S, Darzi A (2002) The relationship between motion analysis and surgical technical assessments. Am J Surg 184:70–73

    Article  PubMed  Google Scholar 

  44. Scott DJ, Rege RV, Bergen PC, Guo WA, Laycock R, Tesfay ST, Valentine RJ, Jones DB (2000) Measuring operative performance after laparoscopic skills training: edited videotape versus direct observation. J LaparoendoscAdv Surg Tech 10:183–190

    Article  CAS  Google Scholar 

  45. Thomas W (2006) Teaching and assessing surgical competence. Ann R Coll Surg 88:429–432

    Article  CAS  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Northumbria Healthcare NHS Trust, North Tyneside General Hospital, Rake Lane, North Shields, NE29 8NH, UK

    Michael G. C. Pellen, Liam F. Horgan, J. Roger Barton & Stephen E. Attwood

  2. Northumbria Upper Gastrointestinal Team of Surgeons, University of Newcastle upon Tyne, The Medical School, Framlington Place, Newcastle upon Tyne, NE2 4HH, UK

    Michael G. C. Pellen, Liam F. Horgan, J. Roger Barton & Stephen E. Attwood

  3. Department of General Surgery, North Tyneside General Hospital, Rake Lane, North Shields, NE29 8NH, UK

    Michael G. C. Pellen

Authors
  1. Michael G. C. Pellen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Liam F. Horgan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. J. Roger Barton
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Stephen E. Attwood
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Michael G. C. Pellen.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Pellen, M.G.C., Horgan, L.F., Barton, J.R. et al. Construct validity of the ProMIS laparoscopic simulator. Surg Endosc 23, 130–139 (2009). https://doi.org/10.1007/s00464-008-0066-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00464-008-0066-y

Keywords

Search

Navigation