Skip to main content
Top

2018 | OriginalPaper | Chapter

Evaluating Surgical Skills from Kinematic Data Using Convolutional Neural Networks

Authors : Hassan Ismail Fawaz, Germain Forestier, Jonathan Weber, Lhassane Idoumghar, Pierre-Alain Muller

Published in: Medical Image Computing and Computer Assisted Intervention – MICCAI 2018

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

The need for automatic surgical skills assessment is increasing, especially because manual feedback from senior surgeons observing junior surgeons is prone to subjectivity and time consuming. Thus, automating surgical skills evaluation is a very important step towards improving surgical practice. In this paper, we designed a Convolutional Neural Network (CNN) to evaluate surgeon skills by extracting patterns in the surgeon motions performed in robotic surgery. The proposed method is validated on the JIGSAWS dataset and achieved very competitive results with 100% accuracy on the suturing and needle passing tasks. While we leveraged from the CNNs efficiency, we also managed to mitigate its black-box effect using class activation map. This feature allows our method to automatically highlight which parts of the surgical task influenced the skill prediction and can be used to explain the classification and to provide personalized feedback to the trainee.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Footnotes
Literature
1.
go back to reference Ahmidi, N., et al.: A dataset and benchmarks for segmentation and recognition of gestures in robotic surgery. IEEE Trans. Biomed. Eng. 64(9), 2025–2041 (2017)CrossRef Ahmidi, N., et al.: A dataset and benchmarks for segmentation and recognition of gestures in robotic surgery. IEEE Trans. Biomed. Eng. 64(9), 2025–2041 (2017)CrossRef
2.
go back to reference Bridgewater, B., et al.: Surgeon specific mortality in adult cardiac surgery: comparison between crude and risk stratified data. Br. Med. J. 327(7405), 13–17 (2003)CrossRef Bridgewater, B., et al.: Surgeon specific mortality in adult cardiac surgery: comparison between crude and risk stratified data. Br. Med. J. 327(7405), 13–17 (2003)CrossRef
3.
go back to reference Fard, M.J., Ameri, S., Darin Ellis, R., Chinnam, R.B., Pandya, A.K., Klein, M.D.: Automated robot-assisted surgical skill evaluation: predictive analytics approach. Int. J. Med. Robot. Comput. Assist. Surg. 14, e1850 (2018) Fard, M.J., Ameri, S., Darin Ellis, R., Chinnam, R.B., Pandya, A.K., Klein, M.D.: Automated robot-assisted surgical skill evaluation: predictive analytics approach. Int. J. Med. Robot. Comput. Assist. Surg. 14, e1850 (2018)
4.
go back to reference Forestier, G., Petitjean, F., Senin, P., Despinoy, F., Jannin, P.: Discovering discriminative and interpretable patterns for surgical motion analysis. In: Artificial Intelligence in Medicine, pp. 136–145 (2017) Forestier, G., Petitjean, F., Senin, P., Despinoy, F., Jannin, P.: Discovering discriminative and interpretable patterns for surgical motion analysis. In: Artificial Intelligence in Medicine, pp. 136–145 (2017)
5.
go back to reference Gao, Y., et al.: The JHU-ISI gesture and skill assessment working set (JIGSAWS): a surgical activity dataset for human motion modeling. In: Modeling and Monitoring of Computer Assisted Interventions, MICCAI Workshop (2014) Gao, Y., et al.: The JHU-ISI gesture and skill assessment working set (JIGSAWS): a surgical activity dataset for human motion modeling. In: Modeling and Monitoring of Computer Assisted Interventions, MICCAI Workshop (2014)
6.
go back to reference Glorot, X., Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks. Int. Conf. Artif. Intell. Stat. 9, 249–256 (2010) Glorot, X., Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks. Int. Conf. Artif. Intell. Stat. 9, 249–256 (2010)
7.
go back to reference Hatala, R., Cook, D.A., Brydges, R., Hawkins, R.: Constructing a validity argument for the objective structured assessment of technical skills (OSATS): a systematic review of validity evidence. Adv. Health Sci. Educ. 20(5), 1149–1175 (2015)CrossRef Hatala, R., Cook, D.A., Brydges, R., Hawkins, R.: Constructing a validity argument for the objective structured assessment of technical skills (OSATS): a systematic review of validity evidence. Adv. Health Sci. Educ. 20(5), 1149–1175 (2015)CrossRef
8.
go back to reference Islam, G., Kahol, K., Li, B., Smith, M., Patel, V.L.: Affordable, web-based surgical skill training and evaluation tool. J. Biomed. Inform. 59, 102–114 (2016)CrossRef Islam, G., Kahol, K., Li, B., Smith, M., Patel, V.L.: Affordable, web-based surgical skill training and evaluation tool. J. Biomed. Inform. 59, 102–114 (2016)CrossRef
9.
go back to reference Kassahun, Y., et al.: Surgical robotics beyond enhanced dexterity instrumentation: a survey of machine learning techniques and their role in intelligent and autonomous surgical actions. Int. J. Comput. Assist. Radiol. Surg. 11(4), 553–568 (2016)CrossRef Kassahun, Y., et al.: Surgical robotics beyond enhanced dexterity instrumentation: a survey of machine learning techniques and their role in intelligent and autonomous surgical actions. Int. J. Comput. Assist. Radiol. Surg. 11(4), 553–568 (2016)CrossRef
10.
go back to reference Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: International Conference on Learning Representations (2015) Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: International Conference on Learning Representations (2015)
11.
go back to reference Maier-Hein, L., et al.: Surgical data science for next-generation interventions. Nat. Biomed. Eng. 1(9), 691–696 (2017)CrossRef Maier-Hein, L., et al.: Surgical data science for next-generation interventions. Nat. Biomed. Eng. 1(9), 691–696 (2017)CrossRef
12.
go back to reference Niitsu, H., et al.: Using the objective structured assessment of technical skills (OSATS) global rating scale to evaluate the skills of surgical trainees in the operating room. Surg. Today 43(3), 271–275 (2013)CrossRef Niitsu, H., et al.: Using the objective structured assessment of technical skills (OSATS) global rating scale to evaluate the skills of surgical trainees in the operating room. Surg. Today 43(3), 271–275 (2013)CrossRef
14.
go back to reference Tedesco, M.M., Pak, J.J., Harris, E.J., Krummel, T.M., Dalman, R.L., Lee, J.T.: Simulation-based endovascular skills assessment: the future of credentialing? J. Vasc. Surg. 47(5), 1008–1014 (2008)CrossRef Tedesco, M.M., Pak, J.J., Harris, E.J., Krummel, T.M., Dalman, R.L., Lee, J.T.: Simulation-based endovascular skills assessment: the future of credentialing? J. Vasc. Surg. 47(5), 1008–1014 (2008)CrossRef
15.
go back to reference Polavarapu, V.: H., Kulaylat, A., Sun, S., Hamed, O.: 100 years of surgical education: the past, present, and future. Bull. Am. Coll. Surg. 98(7), 22–27 (2013) Polavarapu, V.: H., Kulaylat, A., Sun, S., Hamed, O.: 100 years of surgical education: the past, present, and future. Bull. Am. Coll. Surg. 98(7), 22–27 (2013)
16.
go back to reference Vedula, S.S., et al.: Analysis of the structure of surgical activity for a suturing and knot-tying task. Public Libr. Sci. One 11(3), 1–14 (2016) Vedula, S.S., et al.: Analysis of the structure of surgical activity for a suturing and knot-tying task. Public Libr. Sci. One 11(3), 1–14 (2016)
17.
go back to reference Wang, Z., Yan, W., Oates, T.: Time series classification from scratch with deep neural networks: a strong baseline. In: International Joint Conference on Neural Networks, pp. 1578–1585 (2017) Wang, Z., Yan, W., Oates, T.: Time series classification from scratch with deep neural networks: a strong baseline. In: International Joint Conference on Neural Networks, pp. 1578–1585 (2017)
18.
go back to reference Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., Torralba, A.: Learning deep features for discriminative localization. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2921–2929 (2016) Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., Torralba, A.: Learning deep features for discriminative localization. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2921–2929 (2016)
19.
go back to reference Zia, A., Essa, I.: Automated Surgical Skill Assessment in RMIS Training. ArXiv e-prints (2017) Zia, A., Essa, I.: Automated Surgical Skill Assessment in RMIS Training. ArXiv e-prints (2017)
Metadata
Title
Evaluating Surgical Skills from Kinematic Data Using Convolutional Neural Networks
Authors
Hassan Ismail Fawaz
Germain Forestier
Jonathan Weber
Lhassane Idoumghar
Pierre-Alain Muller
Copyright Year
2018
DOI
https://doi.org/10.1007/978-3-030-00937-3_25

Premium Partner