Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
International Journal of Computer Assisted Radiology and Surgery
Published in: International Journal of Computer Assisted Radiology and Surgery 7/2018

04-05-2018 | Original Article

Force estimation from OCT volumes using 3D CNNs

Authors: Nils Gessert, Jens Beringhoff, Christoph Otte, Alexander Schlaefer

Published in: International Journal of Computer Assisted Radiology and Surgery | Issue 7/2018

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Purpose

Estimating the interaction forces of instruments and tissue is of interest, particularly to provide haptic feedback during robot-assisted minimally invasive interventions. Different approaches based on external and integrated force sensors have been proposed. These are hampered by friction, sensor size, and sterilizability. We investigate a novel approach to estimate the force vector directly from optical coherence tomography image volumes.

Methods

We introduce a novel Siamese 3D CNN architecture. The network takes an undeformed reference volume and a deformed sample volume as an input and outputs the three components of the force vector. We employ a deep residual architecture with bottlenecks for increased efficiency. We compare the Siamese approach to methods using difference volumes and two-dimensional projections. Data were generated using a robotic setup to obtain ground-truth force vectors for silicon tissue phantoms as well as porcine tissue.

Results

Our method achieves a mean average error of \({7.7 \pm 4.3}\,{\hbox {mN}}\) when estimating the force vector. Our novel Siamese 3D CNN architecture outperforms single-path methods that achieve a mean average error of \({11.59 \pm 6.7}\,{\hbox {mN}}\). Moreover, the use of volume data leads to significantly higher performance compared to processing only surface information which achieves a mean average error of \({24.38 \pm 22.0}\,{\hbox {mN}}\). Based on the tissue dataset, our methods shows good generalization in between different subjects.

Conclusions

We propose a novel image-based force estimation method using optical coherence tomography. We illustrate that capturing the deformation of subsurface structures substantially improves force estimation. Our approach can provide accurate force estimates in surgical setups when using intraoperative optical coherence tomography.
previous article Laparoscopic ultrasound manipulator with a spring-based elastic mechanism
next article Multistage segmentation model and SVM-ensemble for precise lung nodule detection

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now
Literature
1.
Wilson EB, Bagshahi H, Woodruff VD (2014) Overview of general advantages, limitations, and strategies. In: Robotics in general surgery. Springer, pp 17–22
2.
Kroh M, Chalikonda S (2015) Essentials of robotic surgery. Springer, BerlinCrossRef
3.
4.
De Lorenzo D, De Momi E, Dyagilev I, Manganelli R, Formaglio A, Prattichizzo D, Shoham M, Ferrigno G (2011) Force feedback in a piezoelectric linear actuator for neurosurgery. Int J Med Robot Comput Assist Surg 7(3):268–275
5.
6.
7.
Pacchierotti C, Meli L, Chinello F, Malvezzi M, Prattichizzo D (2015) Cutaneous haptic feedback to ensure the stability of robotic teleoperation systems. Int J Robot Res 34(14):1773–1787CrossRef
8.
Bayle B, Joinie-Maurin M, Barbe L, Gangloff J, De Mathelin M (2014) Robot interaction control in medicine and surgery: original results and open problems. In: Computational surgery and dual training. Springer, pp 169–191
9.
Puangmali P, Liu H, Seneviratne LD, Dasgupta P, Althoefer K (2012) Miniature 3-axis distal force sensor for minimally invasive surgical palpation. IEEE/ASME Trans Mechatron 17(4):646–656CrossRef
10.
Faragasso A, Bimbo J, Noh Y, Jiang A, Sareh S, Liu H, Nanayakkara T, Wurdemann HA, Althoefer K (2014) Novel uniaxial force sensor based on visual information for minimally invasive surgery. In: 2014 IEEE international conference on ICRA, pp 1405–1410
11.
Sokhanvar S, Dargahi J, Najarian S, Arbatani S (2012) Clinical and regulatory challenges for medical devices tactile sensing and displays. Haptic feedback for minimally invasive surgery and robotics
12.
Greminger MA, Nelson BJ (2004) Vision-based force measurement. IEEE Trans Pattern Anal Mach Intell 26(3):290–298CrossRefPubMed
13.
Kim J, Janabi-Sharifi F, Kim J (2010) A haptic interaction method using visual information and physically based modeling. IEEE/ASME Trans Mechatron 15(4):636–645CrossRef
14.
Noohi E, Parastegari S, Žefran M (2014) Using monocular images to estimate interaction forces during minimally invasive surgery. In: 2014 IEEE/RSJ international conference on IROS, pp 4297–4302
15.
Kim W, Seung S, Choi H, Park S, Ko SY, Park JO (2012) Image-based force estimation of deformable tissue using depth map for single-port surgical robot. In: 2012 12th international conference on ICCAS. IEEE, pp 1716–1719
16.
Greminger MA, Nelson BJ (2003) Modeling elastic objects with neural networks for vision-based force measurement. In: 2003 IEEE/RSJ international conference on intelligent robots and systems 2003 (IROS 2003). Proceedings, vol 2. IEEE, pp 1278–1283
17.
Karimirad F, Chauhan S, Shirinzadeh B (2014) Vision-based force measurement using neural networks for biological cell microinjection. J Biomech 47(5):1157–1163CrossRefPubMed
18.
Mozaffari A, Behzadipour S, Kohani M (2014) Identifying the tool-tissue force in robotic laparoscopic surgery using neuro-evolutionary fuzzy systems and a synchronous self-learning hyper level supervisor. Appl Soft Comput 14:12–30CrossRef
19.
Aviles AI, Alsaleh SM, Sobrevilla P, Casals A (2015) Force-feedback sensory substitution using supervised recurrent learning for robotic-assisted surgery. In: 2015 37th annual international conference of the IEEE EMBC, pp 1–4
20.
Aviles AI, Alsaleh SM, Hahn JK, Casals A (2017) Towards retrieving force feedback in robotic-assisted surgery: a supervised neuro-recurrent-vision approach. IEEE Trans Haptics 10(3):431–443CrossRefPubMed
21.
Otte C, Beringhoff J, Latus S, Antoni ST, Rajput O, Schlaefer A (2016) Towards force sensing based on instrument-tissue interaction. In: 2016 IEEE international conference on MFI. IEEE, pp 180–185
22.
Rivero AIA, Alsaleh SM, Hahn JK, Casals A (2016) Towards retrieving force feedback in robotic-assisted surgery: a supervised neuro-recurrent-vision approach. IEEE Transactions on Haptics
23.
Goodfellow I, Bengio Y, Courville A (2016) Deep learning. MIT Press, London
24.
Leal-Taixé L, Canton-Ferrer C, Schindler K (2016) Learning by tracking: Siamese CNN for robust target association. In: Proceedings of the IEEE CVPR workshops, pp 33–40
25.
Zbontar J, LeCun Y (2015) Computing the stereo matching cost with a convolutional neural network. In: Proceedings of the IEEE CVPR, pp 1592–1599
26.
Dosovitskiy A, Fischer P, Ilg E, Hausser P, Hazirbas C, Golkov V, van der Smagt P, Cremers D, Brox T (2015) Flownet: Learning optical flow with convolutional networks. In: Proceedings of the IEEE ICCV, pp 2758–2766
27.
He K, Zhang X, Ren S, Sun J (2016) Identity mappings in deep residual networks. In: ECCV, pp 630–645
28.
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE CVPR, pp 770–778
29.
Glorot X, Bordes A, Bengio Y (2011) Deep sparse rectifier neural networks, vol 15. In: Aistats, p 275
30.
Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. In: ICML, pp 448–456
31.
Kingma D, Ba J (2014) Adam: a method for stochastic optimization. In: ICLR
32.
33.
Abadi M, Agarwal A, Barham P, Brevdo E, Chen Z, Citro C, Corrado GS, Davis A, Dean J, Devin M (2016) Tensorflow: Large-scale machine learning on heterogeneous distributed systems. arXiv preprint arXiv:​1603.​04467
34.
Borchani H, Varando G, Bielza C, Larrañaga P (2015) A survey on multi-output regression. Wiley Interdiscipl Rev Data Min Knowl Discov 5(5):216–233CrossRef
35.
Carrasco-Zevallos O, Keller B, Viehland C, Shen L, Waterman G, Todorich B, Shieh C, Hahn P, Farsiu S, Kuo A, Toth C, Izatt J (2016) Live volumetric (4d) visualization and guidance of in vivo human ophthalmic surgery with intraoperative optical coherence tomography. Sci Rep 6(31):689
36.
Aviles AI, Alsaleh S, Sobrevilla P, Casals A (2016) Exploring the effects of dimensionality reduction in deep networks for force estimation in robotic-assisted surgery. In: SPIE medical imaging. International Society for Optics and Photonics, pp 97,861X–97,861X
37.
Potsaid B, Baumann B, Huang D, Barry S, Cable AE, Schuman JS, Duker JS, Fujimoto JG (2010) Ultrahigh speed 1050nm swept source/fourier domain oct retinal and anterior segment imaging at 100,000 to 400,000 axial scans per second. Opt Express 18(19):20029–20048CrossRefPubMedPubMedCentral
38.
Hillmann D, Franke G, Hinkel L, Bonin T, Koch P, Hüttmann G (2013) Off-axis full-field swept-source optical coherence tomography using holographic refocusing. In: Optical coherence tomography and coherence domain optical methods in biomedicine XVII, vol 8571, p 857104. International Society for Optics and Photonics
39.
Ehlers JP, Srivastava SK, Feiler D, Noonan AI, Rollins AM, Tao YK (2014) Integrative advances for oct-guided ophthalmic surgery and intraoperative oct: microscope integration, surgical instrumentation, and heads-up display surgeon feedback. PLoS One 9(8):e105,224CrossRef
Metadata
Title
Force estimation from OCT volumes using 3D CNNs
Authors
Nils Gessert
Jens Beringhoff
Christoph Otte
Alexander Schlaefer
Publication date
04-05-2018
Publisher
Springer International Publishing
Published in
International Journal of Computer Assisted Radiology and Surgery / Issue 7/2018
Print ISSN: 1861-6410
Electronic ISSN: 1861-6429
DOI
https://doi.org/10.1007/s11548-018-1777-8

Other articles of this Issue 7/2018

International Journal of Computer Assisted Radiology and Surgery 7/2018 Go to the issue

Original Article

Laparoscopic ultrasound manipulator with a spring-based elastic mechanism

Original Article

Multi-ray medical ultrasound simulation without explicit speckle modelling

Original Article

Development of a vibration haptic simulator for shoulder arthroplasty

Original Article

Meaningful Assessment of Robotic Surgical Style using the Wisdom of Crowds

Original Article

In vitro assessment of the accuracy of digital impressions prepared using a single system for full-arch restorations on implants

Original Article

Fully automatic detection of renal cysts in abdominal CT scans

Premium Partner

    Image Credits