Skip to main content
Top
Published in: International Journal of Computer Assisted Radiology and Surgery 5/2014

01-09-2014 | Original Article

Robotic learning of motion using demonstrations and statistical models for surgical simulation

Authors: Tao Yang, Chee Kong Chui, Jiang Liu, Weimin Huang, Yi Su, Stephen K. Y. Chang

Published in: International Journal of Computer Assisted Radiology and Surgery | Issue 5/2014

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Purpose

   In robotic-assisted surgical training, the expertise of surgeons in maneuvering surgical instruments may be utilized to provide the motion trajectories for teaching. However, the motion primitives for trajectory planning are not known until the motion trajectory is generalized. We hypothesize that a generic model that encodes surgical skills using demonstrations and statistical models can be used by the surgical training robot to determine the motion primitive base on the motion trajectory.

Methods

   The generic model was developed from twenty-two sets of motion trajectories of soft tissue division with laparoscopic scissors collected from a robotic laparoscopic surgical training system. Adaptive mean shift method with initial bandwidth determined by the plug-in-rule method was used to identify the primitives in the motion trajectories. Gaussian Mixture Model was applied to model the underlying motion structure. Gaussian Mixture Regression was then applied to reconstruct a generic motion trajectory for the task.

Results

   The generic model and proposed method were investigated in experiments. Motion trajectory of tissue division was model and reconstructed. The motion model which was trained based on primitives determined by adaptive mean shift method produced RMS error of \(3.05^{\circ }\) and \(3.08^{\circ }\) with respect to the demonstrated trajectories of left and right instruments, respectively. The RMS error was smaller than that of k-means method and fixed bandwidth mean shift method. The dexterous features in the demonstrations were also preserved.

Conclusions

   Surgical tasks can be modeled using Gaussian Mixture Model and motion primitives identified by adaptive mean shift method with minimum user intervention. Generic motion trajectory has been successfully reconstructed based on the motion model. Investigation on the effectiveness of this method and generic model for surgical training is ongoing.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Literature
2.
go back to reference Kwartowitz D, Herrell SD, Galloway R (2006) Toward image-guided robotic surgery: determining intrinsic accuracy of the da Vinci robot. Int J Comput Assist Radiol Surg 1(3):157–165. doi:10.1007/s11548-006-0047-3 CrossRef Kwartowitz D, Herrell SD, Galloway R (2006) Toward image-guided robotic surgery: determining intrinsic accuracy of the da Vinci robot. Int J Comput Assist Radiol Surg 1(3):157–165. doi:10.​1007/​s11548-006-0047-3 CrossRef
5.
7.
go back to reference Inamura T, Kojo N, Sonoda T, Sakamoto K, Okada K, Inaba M (2005) Intent imitation using wearable motion capturing system with on-line teaching of task attention. In: 5th IEEE-RAS international conference on humanoid robots, December 5–5 2005, pp 469–474. doi:10.1109/ichr.2005.1573611 Inamura T, Kojo N, Sonoda T, Sakamoto K, Okada K, Inaba M (2005) Intent imitation using wearable motion capturing system with on-line teaching of task attention. In: 5th IEEE-RAS international conference on humanoid robots, December 5–5 2005, pp 469–474. doi:10.​1109/​ichr.​2005.​1573611
8.
go back to reference Calinon S, Guenter F, Billard A (2007) On learning, representing, and generalizing a task in a humanoid robot. IEEE Trans Syst Man Cybern Part B Cybern 37(2):286–298CrossRef Calinon S, Guenter F, Billard A (2007) On learning, representing, and generalizing a task in a humanoid robot. IEEE Trans Syst Man Cybern Part B Cybern 37(2):286–298CrossRef
9.
go back to reference Yamane K, Yamaguchi Y, Nakamura Y (2011) Human motion database with a binary tree and node transition graphs. Auton Robots 30(1):87–98CrossRef Yamane K, Yamaguchi Y, Nakamura Y (2011) Human motion database with a binary tree and node transition graphs. Auton Robots 30(1):87–98CrossRef
10.
go back to reference Reiley CE, Plaku E, Hager GD (2010) Motion generation of robotic surgical tasks: learning from expert demonstrations. In: 2010 annual international conference of the IEEE engineering in medicine and biology society (EMBC), August 31 2010–September 4 2010, pp 967–970 Reiley CE, Plaku E, Hager GD (2010) Motion generation of robotic surgical tasks: learning from expert demonstrations. In: 2010 annual international conference of the IEEE engineering in medicine and biology society (EMBC), August 31 2010–September 4 2010, pp 967–970
11.
go back to reference Niessen W, Viergever M, Thakral A, Wallace J, Tomlin D, Seth N, Thakor N (2001) Surgical motion adaptive robotic technology (S.M.A.R.T): taking the motion out of physiological motion. In: Medical Image Computing and Computer-Assisted Intervention – MICCAI 2001, vol 2208. Lecture Notes in Computer Science. Springer, Berlin Heidelberg, pp 317–325. doi:10.1007/3-540-45468-3_38 Niessen W, Viergever M, Thakral A, Wallace J, Tomlin D, Seth N, Thakor N (2001) Surgical motion adaptive robotic technology (S.M.A.R.T): taking the motion out of physiological motion. In: Medical Image Computing and Computer-Assisted Intervention – MICCAI 2001, vol 2208. Lecture Notes in Computer Science. Springer, Berlin Heidelberg, pp 317–325. doi:10.​1007/​3-540-45468-3_​38
12.
go back to reference Pagador JB, Sanchez-Margallo FM, Sanchez-Peralta LF, Sanchez-Margallo JA, Moyano-Cuevas JL, Enciso-Sanz S, Uson-Gargallo J, Moreno J (2011) Decomposition and analysis of laparoscopic suturing task using tool-motion analysis (TMA): improving the objective assessment. Int J Comput Assist Radiol Surg 7(2):305–313. doi:10.1007/s11548-011-0650-9 PubMedCrossRef Pagador JB, Sanchez-Margallo FM, Sanchez-Peralta LF, Sanchez-Margallo JA, Moyano-Cuevas JL, Enciso-Sanz S, Uson-Gargallo J, Moreno J (2011) Decomposition and analysis of laparoscopic suturing task using tool-motion analysis (TMA): improving the objective assessment. Int J Comput Assist Radiol Surg 7(2):305–313. doi:10.​1007/​s11548-011-0650-9 PubMedCrossRef
14.
go back to reference Hermann M, Faustino G, Daan W, Istvan N, Alois K, Jurgen S (2006) A system for robotic heart surgery that learns to tie knots using recurrent neural networks. In: IEEE/RSJ international conference on intelligent robots and systems, October 2006, pp 543–548. doi:10.1109/iros.2006.282190 Hermann M, Faustino G, Daan W, Istvan N, Alois K, Jurgen S (2006) A system for robotic heart surgery that learns to tie knots using recurrent neural networks. In: IEEE/RSJ international conference on intelligent robots and systems, October 2006, pp 543–548. doi:10.​1109/​iros.​2006.​282190
17.
go back to reference Kormushev P, Calinon S, Caldwell DG (2010) Robot motor skill coordination with EM-based reinforcement learning. In: IEEE/RSJ international conference on intelligent robots and systems (IROS), 18–22 October 2010. pp 3232–3237. doi:10.1109/iros.2010.5649089 Kormushev P, Calinon S, Caldwell DG (2010) Robot motor skill coordination with EM-based reinforcement learning. In: IEEE/RSJ international conference on intelligent robots and systems (IROS), 18–22 October 2010. pp 3232–3237. doi:10.​1109/​iros.​2010.​5649089
18.
go back to reference Thobbi A, Weihua S (2010) Imitation learning of hand gestures and its evaluation for humanoid robots. In: IEEE international conference on information and automation (ICIA), 20–23 June 2010. pp 60–65. doi:10.1109/icinfa.2010.5512333 Thobbi A, Weihua S (2010) Imitation learning of hand gestures and its evaluation for humanoid robots. In: IEEE international conference on information and automation (ICIA), 20–23 June 2010. pp 60–65. doi:10.​1109/​icinfa.​2010.​5512333
19.
go back to reference Reiley CE, Lin HC, Varadarajan B, Vagvolgyi B, Khudanpur S, Yuh DD, Hager GD (2008) Automatic recognition of surgical motions using statistical modeling for capturing variability. Stud Health Technol Inform 132:396–401PubMed Reiley CE, Lin HC, Varadarajan B, Vagvolgyi B, Khudanpur S, Yuh DD, Hager GD (2008) Automatic recognition of surgical motions using statistical modeling for capturing variability. Stud Health Technol Inform 132:396–401PubMed
20.
go back to reference Sheather SJ, Jones MC (1991) A reliable data-based bandwidth selection method for kernel density estimation. J R Stat Soc 53(3):683–690. doi:10.2307/2345597 Sheather SJ, Jones MC (1991) A reliable data-based bandwidth selection method for kernel density estimation. J R Stat Soc 53(3):683–690. doi:10.​2307/​2345597
21.
go back to reference Mugdadi AR, Ahmad IA (2004) A bandwidth selection for kernel density estimation of functions of random variables. Comput Stat Data Anal 47(1):49–62CrossRef Mugdadi AR, Ahmad IA (2004) A bandwidth selection for kernel density estimation of functions of random variables. Comput Stat Data Anal 47(1):49–62CrossRef
22.
go back to reference Horová I, Kolácek J, Zelinka J, Vopatová K (2008) Bandwidth choice for kernel density estimates. In: 6th conference of the asian regional section of the IASC, Yokohama Japan Horová I, Kolácek J, Zelinka J, Vopatová K (2008) Bandwidth choice for kernel density estimates. In: 6th conference of the asian regional section of the IASC, Yokohama Japan
23.
go back to reference Wand MP, Jones MC (1995) Kernel smoothing. Monographs on statistics and applied probability 60. Chapman & Hall, Londong Wand MP, Jones MC (1995) Kernel smoothing. Monographs on statistics and applied probability 60. Chapman & Hall, Londong
24.
go back to reference Comaniciu D, Ramesh V, Meer P (2001) The variable bandwidth mean shift and data-driven scale selection. In: 8th IEEE international conference on computer vision, vol 431, pp 438–445 Comaniciu D, Ramesh V, Meer P (2001) The variable bandwidth mean shift and data-driven scale selection. In: 8th IEEE international conference on computer vision, vol 431, pp 438–445
25.
go back to reference Horová I, Kolácek J, Vopatová K, Full bandwidth matrix selectors for gradient kernel density estimate. Comput Stat Data Anal 57 (1):364–376 Horová I, Kolácek J, Vopatová K, Full bandwidth matrix selectors for gradient kernel density estimate. Comput Stat Data Anal 57 (1):364–376
27.
go back to reference Yang T, Liu J, Huang W, Su Y, Yang L, Chui C, Ang M, Jr., Chang SY (2012) Mechanism of a learning robot manipulator for laparoscopic surgical training. In: Intelligent autonomous systems 12, vol 194. Advances in Intelligent Systems and Computing. Springer, Berlin Heidelberg, pp 17–26. doi:10.1007/978-3-642-33932-5_3 Yang T, Liu J, Huang W, Su Y, Yang L, Chui C, Ang M, Jr., Chang SY (2012) Mechanism of a learning robot manipulator for laparoscopic surgical training. In: Intelligent autonomous systems 12, vol 194. Advances in Intelligent Systems and Computing. Springer, Berlin Heidelberg, pp 17–26. doi:10.​1007/​978-3-642-33932-5_​3
28.
Metadata
Title
Robotic learning of motion using demonstrations and statistical models for surgical simulation
Authors
Tao Yang
Chee Kong Chui
Jiang Liu
Weimin Huang
Yi Su
Stephen K. Y. Chang
Publication date
01-09-2014
Publisher
Springer Berlin Heidelberg
Published in
International Journal of Computer Assisted Radiology and Surgery / Issue 5/2014
Print ISSN: 1861-6410
Electronic ISSN: 1861-6429
DOI
https://doi.org/10.1007/s11548-013-0967-7

Other articles of this Issue 5/2014

International Journal of Computer Assisted Radiology and Surgery 5/2014 Go to the issue

Premium Partner