Abstract
Background
Surgical education relies heavily upon simulation. Assessment tools include robotic simulator assessments and Global Evaluative Assessment of Robotic Skills (GEARS) metrics, which have been validated. Training programs use GEARS for proficiency testing; however, it requires a trained human evaluator. Due to limited time, learners are reliant on surgical simulator feedback to improve their skills. GEARS and simulator scores have been shown to be correlated but in what capacity is unknown. Our goal is to develop a model for predicting GEARS score using simulator metrics.
Methods
Linear and multivariate logistic regressions were used on previously reported data by this group. Subjects performed simple (Ring and Rail 1) and complex (Suture Sponge 1) tasks on simulators, the dV-Trainer (dVT) and the da Vinci Skills Simulator (dVSS). They were scored via simulator metrics and GEARS.
Results
A linear model for each simulator and exercise showed a positive linear correlation. Equations were developed for predicting GEARS Total Score from simulator Overall Score. Next, the effects of each individual simulator metric on the GEARS Total Score for each simulator and exercise were examined. On the dVSS, Excessive Instrument Force was significant for Ring and Rail 1 and Instrument Collision was significant for Suture Sponge 1. On the dVT, Time to Complete was significant for both exercises. Once the significant variables were identified, multivariate models were generated. Comparing the predicted GEARS Total Score from the linear model (using only simulator Overall Score) to that using the multivariate model (using the significant variables for each simulator and exercise), the results were similar.
Conclusions
Our results suggest that trainees can use simulator Overall Score to predict GEARS Total Score using our linear regression equations. This can improve the training process for those preparing for high-stakes assessments.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Riener R (2012) ‘Virtual reality in medicine’ M. Harders. Springer, London
Ramos P, Montez J, Tripp A, Ng CK, Gill IS, Hung AJ (2014) Face, content, construct and concurrent validity of dry laboratory exercises for robotic training using a global assessment tool. BJU Int 113(5):836–842. https://doi.org/10.1111/bju.12559
Hung AJ, Patil MB, Zehnder P, Cai J, Ng CK, Aron M et al (2012) Concurrent and predictive validation of a novel robotic surgery simulator: a prospective, randomized study. J Urol 187(2):630–637. https://doi.org/10.1016/j.juro.2011.09.154
Kiely DJ, Gotlieb WH, Lau S, Zeng X, Samouelian V, Ramanakumar AV et al (2015) Virtual reality robotic surgery simulation curriculum to teach robotic suturing: a randomized controlled trial. J Robot Surg 9(3):179–186
Goh AC, Goldfarb DW, Sander JC, Miles BJ, Dunkin BJ (2012) Global evaluative assessment of robotic skills: validation of a clinical assessment tool to measure robotic surgical skills. J Urol 187(1):247–252. https://doi.org/10.1016/j.juro.2011.09.032
Hung AJ, Bottyan T, Clifford TG, Serang S, Nakhoda ZK, Shah SH et al (2017) Structured learning for robotic surgery utilizing a proficiency score: a pilot study. World J Urol 35(1):27–34. https://doi.org/10.1007/s00345-016-1833-3
Stefanidis D, Sevdalis N, Paige J, Zevin B, Aggarwal R, Grantcharov T et al (2015) Simulation in surgery: what’s needed next? Ann Surg 261(5):846–853. https://doi.org/10.1097/sla.0000000000000826
Dubin AK, Smith R, Julian D, Tanaka A, Mattingly P (2017) A comparison of robotic simulation performance on basic virtual reality skills: simulator subjective vs. objective assessment tools. J Minim Invasive Gynecol. https://doi.org/10.1016/j.jmig.2017.07.019
Intuitive Surgical Inc (2012) Skills Simulator for the da Vinci SI Surgical System
Mimic Technologies Inc (2015) dV-Trainer operators manual
Perrenot C, Perez M, Tran N et al (2012) Surg Endosc 26:2587. https://doi.org/10.1007/s00464-012-2237-0
Acknowledgements
The authors would like to thank Julie Pepe, Ph.D. of Florida Hospital for her help with statistics.
Funding
This study received financial support through a grant from the US Army Telemedicine and Advanced Technology Research Center (TATRC), Grant Number W81XWH-11-2-0158.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Disclosures
Drs. Dubin, Mattingly, Tanaka, and Smith and Ms. Julian have no conflicts of interest or financial ties to disclose.
Rights and permissions
About this article
Cite this article
Dubin, A.K., Julian, D., Tanaka, A. et al. A model for predicting the GEARS score from virtual reality surgical simulator metrics. Surg Endosc 32, 3576–3581 (2018). https://doi.org/10.1007/s00464-018-6082-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00464-018-6082-7