Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
International Journal of Computer Assisted Radiology and Surgery
Published in: International Journal of Computer Assisted Radiology and Surgery 4/2019

18-02-2019 | Original Article

Automatic and near real-time stylistic behavior assessment in robotic surgery

Authors: M. Ershad, R. Rege, Ann Majewicz Fey

Published in: International Journal of Computer Assisted Radiology and Surgery | Issue 4/2019

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Purpose

Automatic skill evaluation is of great importance in surgical robotic training. Extensive research has been done to evaluate surgical skill, and a variety of quantitative metrics have been proposed. However, these methods primarily use expert selected features which may not capture latent information in movement data. In addition, these features are calculated over the entire task time and are provided to the user after the completion of the task. Thus, these quantitative metrics do not provide users with information on how to modify their movements to improve performance in real time. This study focuses on automatic stylistic behavior recognition that has the potential to be implemented in near real time.

Methods

We propose a sparse coding framework for automatic stylistic behavior recognition in short time intervals using only position data from the hands, wrist, elbow, and shoulder. A codebook is built for each stylistic adjective using the positive and negative labels provided for each trial through crowd sourcing. Sparse code coefficients are obtained for short time intervals (0.25 s) in a trial using this codebook. A support vector machine classifier is trained and validated through tenfold cross-validation using the sparse codes from the training set.

Results

The results indicate that the proposed dictionary learning method is able to assess stylistic behavior performance in near real time using user joint position data with improved accuracy compared to using PCA features or raw data.

Conclusion

The possibility to automatically evaluate a trainee’s style of movement in short time intervals could provide the user with online customized feedback and thus improve performance during surgical tasks.
previous article Breast tumor classification using different features of quantitative ultrasound parametric images
next article Wearable technology-based metrics for predicting operator performance during cardiac catheterisation

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now
Literature
1.
Aghdasi N, Bly R, White LW, Hannaford B, Moe K, Lendvay TS (2015) Crowd-sourced assessment of surgical skills in cricothyrotomy procedure. J Surg Res 196(2):302–306CrossRefPubMedPubMedCentral
2.
Aharon M, Elad M, Bruckstein A (2006) rmk-svd: an algorithm for designing overcomplete dictionaries for sparse representation. IEEE Trans Signal Process 54(11):4311–4322CrossRef
3.
Birkmeyer JD, Finks JF, O’reilly A, Oerline M, Carlin AM, Nunn AR, Dimick J, Banerjee M, Birkmeyer NJ (2013) Surgical skill and complication rates after bariatric surgery. N Engl J Med 369(15):1434–1442CrossRefPubMed
4.
Chen C, White L, Kowalewski T, Aggarwal R, Lintott C, Comstock B, Kuksenok K, Aragon C, Holst D, Lendvay T (2014) Crowd-sourced assessment of technical skills: a novel method to evaluate surgical performance. J Surg Res 187(1):65–71CrossRefPubMed
5.
Chen SP, Kirsch S, Zlatev DV, Chang T, Comstock B, Lendvay TS, Liao JC (2016) Optical biopsy of bladder cancer using crowd-sourced assessment. J Am Med Assoc (JAMA) Surg 151(1):90–93
6.
Darzi A, Mackay S (2001) Assessment of surgical competence. BMJ Qual Saf 10(suppl 2):64–69
7.
Datta V, Mackay S, Mandalia M, Darzi A (2001) The use of electromagnetic motion tracking analysis to objectively measure open surgical skill in the laboratory-based model. J Am Coll Surg 193(5):479–485CrossRefPubMed
8.
Datta V, Chang A, Mackay S, Darzi A (2002) The relationship between motion analysis and surgical technical assessments. Am J Surg 184(1):70–73CrossRefPubMed
9.
Deal SB, Lendvay TS, Haque MI, Brand T, Comstock B, Warren J, Alseidi A (2016) Crowd-sourced assessment of technical skills: an opportunity for improvement in the assessment of laparoscopic surgical skills. Am J Surg 211(2):398–404CrossRefPubMed
10.
Derossis AM, Fried GM, Abrahamowicz M, Sigman HH, Barkun JS, Meakins JL (1998) Development of a model for training and evaluation of laparoscopic skills. Am J Surg 175(6):482–487CrossRefPubMed
11.
Dhamala M, Rangarajan G, Ding M (2008) Estimating granger causality from Fourier and wavelet transforms of time series data. Phys Rev Lett 100(1):018701CrossRefPubMed
12.
Efron B, Hastie T, Johnstone I, Tibshirani R (2004) Least angle regression. Ann Stat 32(2):407–499CrossRef
13.
El Moudden I, Ouzir M, Benyacoub B, ElBernoussi S (2016) Mining human activity using dimensionality reduction and pattern recognition. Contemp Eng Sci: CES 9:21CrossRef
14.
Ershad M, Koesters Z, Rege R, Majewicz A (2016) Meaningful assessment of surgical expertise: semantic labeling with data and crowds. In: International conference on medical image computing and computer-assisted intervention (MICCAI). Springer, pp 508–515
15.
Ershad M, Rege R, Fey AM (2018a) Meaningful assessment of robotic surgical style using the wisdom of crowds. Int J Comput Assist Radiol Surg: IJCARS, 1–12
16.
Ershad M, Rege R, Majewicz A (2018b) Surgical skill level assessment using automatic feature extraction methods. In: Medical imaging: image-guided procedures, robotic interventions, and modeling. International Society for Optics and Photonics, p 6
17.
Fard MJ, Ameri S, Darin Ellis R, Chinnam RB, Pandya AK, Klein MD (2018) Automated robot-assisted surgical skill evaluation: predictive analytics approach. Int J Med Robot Comput Assist Surg 14(1):e1850CrossRef
18.
19.
20.
21.
Holst D, Kowalewski TM, White LW, Brand TC, Harper JD, Sorenson MD, Kirsch S, Lendvay TS (2015) Crowd-sourced assessment of technical skills: an adjunct to urology resident surgical simulation training. J Endourol 29(5):604–609CrossRefPubMed
22.
Hoyer PO (2002) Non-negative sparse coding. In: Proceedings of the 12th IEEE workshop on neural networks for signal processing. IEEE, pp 557–565
23.
Hung AJ, Chen J, Che Z, Nilanon T, Jarc A, Titus M, Oh PJ, Gill IS, Liu Y (2018) Utilizing machine learning and automated performance metrics to evaluate robot-assisted radical prostatectomy performance and predict outcomes. J Endourol 32(5):438–444CrossRefPubMed
24.
Jin A, Yeung S, Jopling J, Krause J, Azagury D, Milstein A, Fei-Fei L (2018) Tool detection and operative skill assessment in surgical videos using region-based convolutional neural networks. arXiv preprint arXiv:​1802.​08774
25.
Karg M, Jenke R, Seiberl W, Kühnlenz K, Schwirtz A, Buss M (2009) A comparison of pca, kpca and lda for feature extraction to recognize affect in gait kinematics. In: 3rd IEEE international conference on affective computing and intelligent interaction and workshops (ACII). IEEE, pp 1–6
26.
Kazanzides P, Chen Z, Deguet A, Fischer GS, Taylor RH, DiMaio SP (2014) An open-source research kit for the da vinci® surgical system. In: 2014 IEEE international conference on robotics and automation (ICRA). IEEE, pp 6434–6439
27.
Kirkwood RN, Resende RA, Magalhães C, Gomes HA, Mingoti SA, Sampaio RF (2011) Application of principal component analysis on gait kinematics in elderly women with knee osteoarthritis. Braz J Phys Ther 15(1):52–58CrossRef
28.
Kowalewski TM, Comstock B, Sweet R, Schaffhausen C, Menhadji A, Averch T, Box G, Brand T, Ferrandino M, Kaouk J (2016) Crowd-sourced assessment of technical skills for validation of basic laparoscopic urologic skills tasks. J Urol 195(6):1859–1865CrossRefPubMed
29.
Law H, Ghani K, Deng J (2017) Surgeon technical skill assessment using computer vision based analysis. In: Machine learning for healthcare conference, pp 88–99
30.
Liang K, Xing Y, Li J, Wang S, Li A, Li J (2018) Motion control skill assessment based on kinematic analysis of robotic end-effector movements. Int J Med Robot Comput Assist Surg 14(1):e1845CrossRef
31.
Maier-Hein L, Vedula SS, Speidel S, Navab N, Kikinis R, Park A, Eisenmann M, Feu-ssner H, Forestier G, Giannarou S, Hashizume M, Katic D, Kenngott H, Kranzfelder M, Malpani A, März K, Neumuth T, Padoy N, Pugh CM, Schoch N, Stoyanov D, Taylor R, Wagner M, Hager GD, Jannin P (2017) Surgical data science for next-generation interventions. Nat Biomed Eng 1:691–696CrossRefPubMed
32.
Mallat SG (1989) A theory for multiresolution signal decomposition: the wavelet representation. IEEE Trans Pattern Anal Mach Intell 11(7):674–693CrossRef
33.
34.
35.
Milovanović I, Popović DB (2012) Principal component analysis of gait kinematics data in acute and chronic stroke patients. Comput Math Methods Med 2012:8
36.
Nisky I, Hsieh MH, Okamura AM (2013) A framework for analysis of surgeon arm posture variability in robot-assisted surgery. In: IEEE international conference on robotics and automation (ICRA). IEEE, pp 245–251
37.
Nisky I, Hsieh MH, Okamura AM (2014) Uncontrolled manifold analysis of arm joint angle variability during robotic teleoperation and freehand movement of surgeons and novices. IEEE Trans Biomed Eng 61(12):2869–2881CrossRefPubMed
38.
Polin MR, Siddiqui NY, Comstock BA, Hesham H, Brown C, Lendvay TS, Martino MA (2016) Crowdsourcing: a valid alternative to expert evaluation of robotic surgery skills. Am J Obstet Gynecol 215(5):644–e1CrossRefPubMed
39.
Reiley CE, Hager GD (2009) Task versus subtask surgical skill evaluation of robotic minimally invasive surgery. In: International conference on medical image computing and computer-assisted intervention (MICCAI). Springer, pp 435–442
40.
Reiley CE, Lin HC, Varadarajan B, Vagvolgyi B, Khudanpur S, Yuh DD, Hager GD (2008) Automatic recognition of surgical motions using statistical modeling for capturing variability. Stud Health Technol Inform 132(1):396–401PubMed
41.
42.
Smith S, Torkington J, Brown T, Taffinder N, Darzi A (2002) Motion analysis. Surg Endosc 16(4):640–645CrossRefPubMed
43.
Varadarajan B, Reiley C, Lin H, Khudanpur S, Hager G (2009) Data-derived models for segmentation with application to surgical assessment and training. In: International conference on medical image computing and computer-assisted intervention (MICCAI). Springer, pp 426–434
44.
45.
46.
White LW, Kowalewski TM, Dockter RL, Comstock B, Hannaford B, Lendvay TS (2015) Crowd-sourced assessment of technical skill: a valid method for discriminating basic robotic surgery skills. J Endourol 29(11):1295–1301CrossRefPubMed
47.
Wright J, Yang AY, Ganesh A, Sastry SS, Ma Y (2009) Robust face recognition via sparse representation. IEEE Trans Pattern Anal Mach Intell 31(2):210–227CrossRefPubMed
Metadata
Title
Automatic and near real-time stylistic behavior assessment in robotic surgery
Authors
M. Ershad
R. Rege
Ann Majewicz Fey
Publication date
18-02-2019
Publisher
Springer International Publishing
Published in
International Journal of Computer Assisted Radiology and Surgery / Issue 4/2019
Print ISSN: 1861-6410
Electronic ISSN: 1861-6429
DOI
https://doi.org/10.1007/s11548-019-01920-6

Other articles of this Issue 4/2019

International Journal of Computer Assisted Radiology and Surgery 4/2019 Go to the issue

Original Article

Early esophageal adenocarcinoma detection using deep learning methods

Original Article

Predicting underestimation of ductal carcinoma in situ: a comparison between radiomics and conventional approaches

Original Article

An analytical approach for the simulation of realistic low-dose fluoroscopic images

Original Article

“Deep-Onto” network for surgical workflow and context recognition

Original Article

Temporal enhancement of 2D color Doppler echocardiography sequences by fragment-based frame reordering and refinement

Original Article

Joint reconstruction and classification of tumor cells and cell interactions in melanoma tissue sections with synthesized training data

Premium Partner

    Image Credits