Skip to main content
SpringerLink
Log in

Development of force-based metrics for skills assessment in minimally invasive surgery

  • Published:
Surgical Endoscopy Aims and scope Submit manuscript

Abstract

Background

The loss of haptic information that results from the reduced-access conditions present in minimally invasive surgery (MIS) may compromise the safety of the procedures. This limitation must be overcome through training. However, current methods for determining the skill level of trainees do not measure critical elements of skill attainment. This study aimed to evaluate the usefulness of force information for the assessment of skill during MIS.

Methods

To achieve the study goal, experiments were performed using a set of sensorized instruments capable of measuring instrument position and tissue interaction forces. Several force-based metrics were developed as well as metrics that combine force and position information.

Results

The results show that experience level has a strong correlation with the new force-based metrics presented in this article. In particular, the integral and the derivative of the forces or the metrics that combine force and position provide the strongest correlations.

Conclusions

This study showed that force-based metrics are better indications of performance than metrics based on task completion time or position information alone. The proposed metrics can be automatically computed, are completely objective, and measure important aspects of performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Wulf G, Shea C, Lewthwaite R (2010) Motor skill learning and performance: a review of influential factors. Med Educ 44:75–84

    Article  PubMed  Google Scholar 

  2. Feldman LS, Hagarty SE, Ghitulescu G, Stanbridge D, Fried GM (2004) Relationship between objective assessment of technical skills and subjective in-training evaluation in surgical residents. J Am Coll Surg 198:105–110

    Article  PubMed  Google Scholar 

  3. Cavallo F, Megali G, Sinigaglia S, Tonet O, Dario P (2006) A biomedical analysis of a surgeon’s gesture in a laparoscopic virtual scenario. In: Westwood JD, Haluck RS, Hoffman HM, Mogel GT, Phillips R, Robb RA, Vosburgh KG (eds) Medicine meets virtual reality. Studies in health technology and informatics, vol 119. IOS Press, Long Beach, pp 79–84

  4. Dayan AB, Ziv A, Berkenstadt H, Munz Y (2008) A simple, low-cost platform for basic laparoscopic skills training. Surg Innov 15:136–142

    Article  PubMed  Google Scholar 

  5. Schreuder HWR, Oei G, Maas M, Borleffs JCC, Schijven MP (2011) Implementation of simulation in surgical practice: minimally invasive surgery has taken the lead: the Dutch experience. Med Teach 33:105–115

    Article  PubMed  Google Scholar 

  6. Khan MS, Bann SD, Darzi A, Butler PEM (2003) Assessing surgical skill. Plast Reconstr Surg 112:1886–1889

    Article  CAS  PubMed  Google Scholar 

  7. Elliott D, Hansen S, Lyons J, Grierson LEM, Bennett SJ, Hayes SJ (2010) Goal-directed aiming: two components but multiple processes. Psychol Bull 136:1023–1044

    Article  PubMed  Google Scholar 

  8. Elliott D, Helsen WF, Chua R (2001) A century later: Woodworth’s (1899) two-component model of goal-directed aiming. Psychol Bull 127:342–357

    Article  CAS  PubMed  Google Scholar 

  9. Elliott D, Grierson LEM, Hayes SJ, Lyons J, Elliott D (2011) Action representations in perception, motor control, and learning: implications for medical education. Med Educ 45:119–131

    Article  PubMed  Google Scholar 

  10. Choi Y, Qi F, Gordon J, Schweighofer N (2008) Performance-based adaptive schedules enhance motor learning. J Mot Behav 40:273–280

    Article  PubMed  Google Scholar 

  11. Reiley C, Lin H, Yuh D, Hager G (2011) Review of methods for objective surgical skill evaluation. Surg Endosc 25:356–366

    Article  PubMed  Google Scholar 

  12. Goff B, Mandel L, Lentz G, Vanblaricom A, Oelschlager A-MA, Lee D, Galakatos A, Davies M, Nielsen P (2005) Assessment of resident surgical skills: is testing feasible? Am J Obstet Gynecol 192:1331–1338

    Article  PubMed  Google Scholar 

  13. Lammers RL, Davenport M, Korley F, Griswold-Theodorson S, Fitch MT, Narang AT, Evans LV, Gross A, Rodriguez E, Dodge KL, Hamann CJ, Robey WC (2008) Teaching and assessing procedural skills using simulation: metrics and methodology. Acad Emerg Med 15:1079–1087

    Article  PubMed  Google Scholar 

  14. Moody L, Baber C, Arvanitis TN, Elliott M (2003) Objective metrics for the evaluation of simple surgical skills in real and virtual domains. Presence Teleoper Virtual Environ 12:207–221

    Article  Google Scholar 

  15. Datta V, Mandalia M, Mackay S, Chang A, Cheshire N, Darzi A (2002) Relationship between skill and outcome in the laboratory-based model. Surgery 131:318–323

    Article  PubMed  Google Scholar 

  16. Dubrowski A, Sidhu R, Park J, Carnahan H (2005) Quantification of motion characteristics and forces applied to tissues during suturing. Am J Surg 190:131–136

    Article  PubMed  Google Scholar 

  17. Beyer L, Troyer JD, Mancini J, Bladou F, Berdah SV, Karsenty G (2011) Impact of laparoscopy simulator training on the technical skills of future surgeons in the operating room: a prospective study. Am J Surg 202:265–272

    Article  PubMed  Google Scholar 

  18. Derossis AM, Fried GM, Abrahamowicz M, Sigman HH, Barkun JS, Meakins JL (1998) Development of a model for training and evaluations of laparoscopic skills. Am J Surg 175:482–487

    Article  CAS  PubMed  Google Scholar 

  19. Bann S, Kwok K-F, Lo C-Y, Darzi A, Wong J (2003) Objective assessment of technical skills of surgical trainees in Hong Kong. Br J Surg 90:1294–1299

    Article  CAS  PubMed  Google Scholar 

  20. Cotin S, Stylopoulos N, Ottensmeyer M, Neumann P, Bardsley R, Dawson S (2005) Surgical training system for laparoscopic procedures. US Patent US2005/0 142 525 A1, 30 June

  21. Oropesa I, Lamata P, Sanchez-Gonzalez P, Pagador J, Garcia M, Sanchez-Margallo F, Gomez E (2011) Virtual reality simulators for objective evaluation on laparoscopic surgery: current trends and benefits. In: Kim J (ed) Virtual reality. Intech Open Access Publisher, Rijeka, pp 349–374

    Google Scholar 

  22. Cao C, Zhou M, Jones D, Schwaitzberg S (2007) Can surgeons think and operate with haptics at the same time? J Gastrointest Surg 11:1564–1569

    Article  PubMed  Google Scholar 

  23. Datta V, Mackay S, Mandalia M, Darzi A (2001) The use of electromagnetic motion tracking analysis to objectively measure open surgical skill in the laboratory-based model. J Am Coll Surg 193:479–485

    Article  CAS  PubMed  Google Scholar 

  24. Rosen J, Hannaford B (2001) Markov modeling of minimally invasive surgery based on tool/tissue interaction and force/torque signatures for evaluating surgical skills”. IEEE Trans Biomed Eng 48:579–591

    Article  CAS  PubMed  Google Scholar 

  25. Boyle E, Al-Akash M, Gallagher AG, Traynor O, Hill ADK, Neary PC (2011) Optimising surgical training: use of feedback to reduce errors during a simulated surgical procedure. Postgrad Med J 87:524–528

    Article  PubMed  Google Scholar 

  26. Takada K, Yashiro K, Takagi M (2006) Reliability and sensitivity of jerk-cost measurement for evaluating irregularity of chewing jaw movements. Physiol Meas 27:609–622

    Article  PubMed  Google Scholar 

  27. Megali G, Sinigaglia S, Tonet O, Dario P (2006) Modelling and evaluation of surgical performance using hidden Markov models. IEEE Trans Biomed Eng 53:1911–1919

    Article  PubMed  Google Scholar 

  28. Loukas CG, Georgiou E (2011) Multivariate autoregressive modeling of hand kinematics for laparoscopic skills assessment of surgical trainees. IEEE Trans Biomed Eng 58:3289–3297. http://dblp.uni-trier.de/db/journals/tbe/tbe58.html#LoukasG11

    Google Scholar 

  29. Rohrer B, Fasoli S, Krebs H, Hughes R, Volpe B, Frontera W, Stein J, Hogan N (2002) Movement smoothness changes during stroke recovery. J Neurosci 22:8297–8304

    CAS  PubMed  Google Scholar 

  30. Cristancho S, Hodgson A, Panton O, Meneghetti A, Warnock G, Qayumi K (2009) Intraoperative monitoring of laparoscopic skill development based on quantitative measures. Surg Endosc 23:2181–2190

    Article  PubMed  Google Scholar 

  31. Hwang H, Lim J, Kinnaird C, Nagy A, Panton O, Hodgson A, Qayumi K (2006) Correlating motor performance with surgical error in laparoscopic cholecystectomy. Surg Endosc 20:651–655

    Article  CAS  PubMed  Google Scholar 

  32. Hogan N, Sternad D (2009) Sensitivity of smoothness measures to movement duration, amplitude, and arrests. J Mot Behav 41:529–534

    Article  PubMed Central  PubMed  Google Scholar 

  33. Tang B, Hanna G, Cuschieri A (2005) Analysis of errors enacted by surgical trainees during skills training courses. Surgery 138:14–20

    Article  CAS  PubMed  Google Scholar 

  34. Stylopoulos N, Vosburgh KG (2007) Assessing technical skill in surgery and endoscopy: a set of metrics and an algorithm (C-PASS) to assess skills in surgical and endoscopic procedures. Surg Innov 14:113–121

    Article  PubMed  Google Scholar 

  35. Trejos AL, Patel RV, Naish MD, Malthaner RA, Schlachta CM (2013) The application of force sensing to skills assessment in minimally invasive surgery. IEEE International Conference on Robotics and Automation, Karlsruhe, 6–10 May

  36. Trejos AL, Patel RV, Naish MD, Schlachta CM (2009) A sensorized instrument for skills assessment and training in minimally invasive surgery. ASME J Med Devices 3:041 002-1–041 002-12

    Article  Google Scholar 

  37. Trejos AL (2012) A sensorized instrument for minimally invasive surgery for the measurement of forces during training and surgery: development and applications. PhD Dissertation, Department of Electrical and Computer Engineering, The University of Western Ontario, Canada

  38. Stefanidis D, Hope WW, Korndorffer JR, Markley S, Scott DJ (2010) Initial laparoscopic basic skills training shortens the learning curve of laparoscopic suturing and is cost-effective. J Am Coll Surg 210:436–440

    Article  PubMed  Google Scholar 

Download references

Acknowledgments

The authors thank Dr. Michael Naish, Andrew Lyle, Abelardo Escoto, Chris Ward, Kevin Barker, Jerry Dafoe, and Dr. Ives Bureau for their assistance with several aspects of the work presented in this article and all the surgeons and trainees who willingly participated in the experiments. The support from Quanser, Inc., Markham, Ontario, Canada, is gratefully acknowledged. was supported by a grant from the Western Innovation Fund (R.V. Patel); by the Natural Sciences and Engineering Research Council (NSERC) of Canada under grants I2IPJ 363985-07 and RGPIN-1345 (R.V. Patel); by the Canada Research Chairs Program (R.V. Patel); and by infrastructure grants from the Canada Foundation for Innovation awarded to the London Health Sciences Centre (Canadian Surgical Technologies & Advanced Robotics [CSTAR]) and to The University of Western Ontario (UWO) (R.V. Patel). Financial support for Dr. Trejos was provided by an NSERC Alexander Graham Bell Canada Graduate Scholarship.

Disclosures

Ana Luisa Trejos and Rajni V. Patel are inventors and have a patent application for the sensorized surgical instruments. The performance metrics presented in this article are not part of the patent application. They have no other conflicts of interest and no financial ties to disclose. Richard A. Malthaner and Christopher M. Schlachta have no conflicts of interest or financial ties to disclose.

Author information

Authors and Affiliations

  1. Canadian Surgical Technologies and Advanced Robotics, Lawson Health Research Institute, 339 Windermere Road, London, ON, Canada

    Ana Luisa Trejos, Rajni V. Patel, Richard A. Malthaner & Christopher M. Schlachta

  2. Department of Electrical and Computer Engineering, The University of Western Ontario, London, ON, Canada

    Ana Luisa Trejos & Rajni V. Patel

  3. Department of Surgery, The University of Western Ontario, London, ON, Canada

    Rajni V. Patel, Richard A. Malthaner & Christopher M. Schlachta

  4. Department of Epidemiology and Biostatistics, The University of Western Ontario, London, ON, Canada

    Richard A. Malthaner

Authors
  1. Ana Luisa Trejos
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rajni V. Patel
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Richard A. Malthaner
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Christopher M. Schlachta
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ana Luisa Trejos.

Additional information

Rajni V. Patel is the Project leader.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Trejos, A.L., Patel, R.V., Malthaner, R.A. et al. Development of force-based metrics for skills assessment in minimally invasive surgery. Surg Endosc 28, 2106–2119 (2014). https://doi.org/10.1007/s00464-014-3442-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00464-014-3442-9

Keywords

Search

Navigation