Skip to main content

Advertisement

SpringerLink
Log in

Toward Improving Safety in Neurosurgery with an Active Handheld Instrument

  • Medical Robotics
  • Published:
Annals of Biomedical Engineering Aims and scope Submit manuscript

Abstract

Microsurgical procedures, such as petroclival meningioma resection, require careful surgical actions in order to remove tumor tissue, while avoiding brain and vessel damaging. Such procedures are currently performed under microscope magnification. Robotic tools are emerging in order to filter surgeons’ unintended movements and prevent tools from entering forbidden regions such as vascular structures. The present work investigates the use of a handheld robotic tool (Micron) to automate vessel avoidance in microsurgery. In particular, we focused on vessel segmentation, implementing a deep-learning-based segmentation strategy in microscopy images, and its integration with a feature-based passive 3D reconstruction algorithm to obtain accurate and robust vessel position. We then implemented a virtual-fixture-based strategy to control the handheld robotic tool and perform vessel avoidance. Clay vascular phantoms, lying on a background obtained from microscopy images recorded during petroclival meningioma surgery, were used for testing the segmentation and control algorithms. When testing the segmentation algorithm on 100 different phantom images, a median Dice similarity coefficient equal to 0.96 was achieved. A set of 25 Micron trials of 80 s in duration, each involving the interaction of Micron with a different vascular phantom, were recorded, with a safety distance equal to 2 mm, which was comparable to the median vessel diameter. Micron’s tip entered the forbidden region 24% of the time when the control algorithm was active. However, the median penetration depth was 16.9 μm, which was two orders of magnitude lower than median vessel diameter. Results suggest the system can assist surgeons in performing safe vessel avoidance during neurosurgical procedures.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
Figure 7
Figure 8
Figure 9
Figure 10
Figure 11
Figure 12
Figure 13

Similar content being viewed by others

Notes

  1. https://www.opencv.org/.

References

  1. Abadi, M., A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, G. S. Corrado, A. Davis, J. Dean, M. Devin, et al. Tensorflow: Large-scale machine learning on heterogeneous distributed systems. arXiv preprint, 2016. arXiv: 1603.04467.

  2. Al-Mefty, O., J. L. Fox, Sr., and R. R. Smith. Petrosal approach for petroclival meningiomas. Neurosurgery 22(3):510–517, 1988.

    Article  CAS  Google Scholar 

  3. Bay, H., T. Tuytelaars, and L. Van Gool. SURF: Speeded up robust features. European Conference on Computer Vision, 2006, pp. 404–417.

  4. Becker, B. C., R. A. MacLachlan, L. A. Lobes, G. D. Hager, and C. N. Riviere. Vision-based control of a handheld surgical micromanipulator with virtual fixtures. IEEE Trans. Robot. 29(3):674–683, 2013.

    Article  Google Scholar 

  5. Beretta, E., E. De Momi, F. Rodriguez y Baena, and G. Ferrigno. Adaptive hands-on control for reaching and targeting tasks in surgery. Int. J. Adv. Robot. Syst. 12(5):50, 2015.

    Article  Google Scholar 

  6. Beretta, E., G. Ferrigno, and E. De Momi. Nonlinear force feedback enhancement for cooperative robotic neurosurgery enforces virtual boundaries on cortex surface. J. Med. Robot. Res. 1(02):1650001, 2016.

  7. Braun, D., S. Yang, J. N. Martel, C. N. Riviere, and B. C. Becker. EyeSLAM: real-time simultaneous localization and mapping of retinal vessels during intraocular microsurgery. Int. J. Med. Robot. Comput. Assist. Surg. 14(1):e1848, 2017.

    Article  Google Scholar 

  8. Chan, T. F., and L. A. Vese. Active contours without edges. Trans. Image Process. 10(2):266–277, 2001.

    Article  CAS  Google Scholar 

  9. Chaudhuri, S., S. Chatterjee, N. Katz, M. Nelson, and M. Goldbaum. Detection of blood vessels in retinal images using two-dimensional matched filters. IEEE Trans. Med. Imaging 8(3):263–269, 1989.

    Article  CAS  Google Scholar 

  10. Cheng, Y., X. Hu, J. Wang, Y. Wang and S. Tamura. Accurate vessel segmentation with constrained B-snake. IEEE Trans. Med. Imaging 24(8):2440–2455, 2015.

    Article  Google Scholar 

  11. Dasgupta, A., and S. Singh. A fully convolutional neural network based structured prediction approach towards the retinal vessel segmentation. In: IEEE International Symposium on Biomedical Imaging, 2017, pp. 248–251.

  12. DiLuna, M. L., and K. R. Bulsara. Surgery for petroclival meningiomas: a comprehensive review of outcomes in the skull base surgery era. Skull Base 20(05):337–342, 2010.

    Article  Google Scholar 

  13. Durrant-Whyte, H., and T. Bailey. Simultaneous localization and mapping: part I. IEEE Robot. Autom. Mag. 13(2):99–110, 2006.

    Article  Google Scholar 

  14. El-Manadili, Y., and K. Novak. Precision rectification of SPOT imagery using the direct linear transformation model. Photogram. Eng. Remot. Sens. 62(1):67–72, 1996.

    Google Scholar 

  15. Esteva, A., B. Kuprel, R. A. Novoa, J. Ko, S. M. Swetter, H. M. Blau, and S. Thrun: Dermatologist-level classification of skin cancer with deep neural networks. Nature 542(7639):115–118, 2017.

    Article  CAS  Google Scholar 

  16. Felzenszwalb, P., and D. Huttenlocher. Distance transforms of sampled functions. Tech. Rep., Cornell University, 2004.

  17. Frangi, A. F., W. J. Niessen, K. L. Vincken, and M. A. Viergever. Multiscale vessel enhancement filtering. International Conference on Medical Image Computing and Computer-Assisted Intervention. Berlin: Springer, 1998, pp. 130–137.

  18. Fraz, M. M., P. Remagnino, A. Hoppe, B. Uyyanonvara, A. R. Rudnicka, C. G. Owen, and S. A. Barman. Blood vessel segmentation methodologies in retinal images—a survey. Comput. Methods Progr. Biomed. 108(1):407–433, 2012.

    Article  CAS  Google Scholar 

  19. Fu, H., Wang, C., Tao, D., Black, M.J.: Occlusion boundary detection via deep exploration of context. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 241–250 (2016)

  20. Fu, H., Y. Xu, S. Lin, D. W. K. Wong, and J. Liu. DeepVessel: retinal vessel segmentation via deep learning and conditional random field. International Conference on Medical Image Computing and Computer-Assisted Intervention. Berlin: Springer, 2016, pp. 132–139.

  21. Ganin, Y., and V. Lempitsky. N4-fields: neural network nearest neighbor fields for image transforms. Asian Conference on Computer Vision. Berlin: Springer, 2014, pp. 536–551.

  22. Gijbels, A., E. B. Vander Poorten, P. Stalmans, and D. Reynaerts. Development and experimental validation of a force sensing needle for robotically assisted retinal vein cannulations. IEEE International Conference on Robotics and Automation, pp. 2270–2276, 2015.

  23. Girshick, R., J. Donahue, T. Darrell, and J. Malik. Rich feature hierarchies for accurate object detection and semantic segmentation. Conference on Computer Vision and Pattern Recognition, pp. 580–587, 2014.

  24. Gonenc, B., M. A. Balicki, J. Handa, P. Gehlbach, C. N. Riviere, R. H. Taylor, and I. Iordachita. Preliminary evaluation of a micro-force sensing handheld robot for vitreoretinal surgery. International Conference on Intelligent Robots and Systems, pp. 4125–4130, 2012.

  25. Hackethal, A., M. Koppan, K. Eskef, and H. R. Tinneberg. Handheld articulating laparoscopic instruments driven by robotic technology. First clinical experience in gynecological surgery. Gynecol. Surg. 9(2):203, 2011.

    Article  Google Scholar 

  26. Kingma, D., and J. Ba. Adam: a method for stochastic optimization. arXiv preprint, 2014. arXiv: 1412.6980.

  27. Krizhevsky, A., I. Sutskever, and G. E. Hinton. Imagenet classification with deep convolutional neural networks. Adv. Neural Inf. Process. Syst. 25:1097–1105, 2012.

  28. Kwok, K. W., K. H. Tsoi, V. Vitiello, J. Clark, G. C. Chow, W. Luk, and G. Z. Yang. Dimensionality reduction in controlling articulated snake robot for endoscopy under dynamic active constraints. IEEE Trans. Robot. 29(1):15–31, 2013.

    Article  Google Scholar 

  29. Lam, L., S. W. Lee, and C. Y. Suen. Thinning methodologies—a comprehensive survey. IEEE Trans. Pattern Anal. Mach. Intell. 14(9):869–885, 1992.

    Article  Google Scholar 

  30. Lang, J. Clinical Anatomy of the Head: Neurocranium, Orbit, Craniocervical Regions. Berlin: Springer, 2012.

  31. Lawrence, J. D., A. M. Frederickson, Y. F. Chang, P. M. Weiss, P. C. Gerszten, and R. F. Sekula, Jr. An investigation into quality of life improvement in patients undergoing microvascular decompression for hemifacial spasm. J. Neurosurg. 128(1):193–201, 2017.

    Article  Google Scholar 

  32. LeCun, Y., Y. Bengio, and G. Hinton. Deep learning. Nature 521(7553):436, 2015.

    Article  CAS  Google Scholar 

  33. Lee, K. H., Z. Guo, G. C. Chow, Y. Chen, W. Luk, and K. W. Kwok. GPU-based proximity query processing on unstructured triangular mesh model. IEEE International Conference on Robotics and Automation, 2015, pp. 4405–4411.

  34. Li, Q., B. Feng, L. Xie, P. Liang, H. Zhang, and T. Wang. A cross-modality learning approach for vessel segmentation in retinal images. IEEE Trans. Med. Imaging 35(1):109–118, 2016.

    Article  Google Scholar 

  35. Liskowski, P., and K. Krawiec. Segmenting retinal blood vessels with deep neural networks. IEEE Trans. Med. Imaging 35(11):2369–2380, 2016.

    Article  Google Scholar 

  36. Liu, N., H. Li, M. Zhang, J. Liu, Z. Sun, and T. Tan. Accurate iris segmentation in non-cooperative environments using fully convolutional networks. IEEE International Conference on Biometrics, 2016, pp. 1–8.

  37. Lonner, J. H. Robotically assisted unicompartmental knee arthroplasty with a handheld image-free sculpting tool. Orthop. Clin. N. Am. 47(1):29–40, 2016.

    Article  Google Scholar 

  38. MacLachlan, R. A., B. C. Becker, J. C. Tabarés, G. W. Podnar, L. A. Lobes, Jr, and C. N. Riviere. Micron: an actively stabilized handheld tool for microsurgery. IEEE Trans. Robot. 28(1):195–212, 2012.

    Article  Google Scholar 

  39. MacLachlan, R. A., and C. N. Riviere. High-speed microscale optical tracking using digital frequency-domain multiplexing. IEEE Trans. Instrum. Meas. 58(6):1991–2001, 2009.

    Article  Google Scholar 

  40. Maninis, K. K., J. Pont-Tuset, P. Arbeláez, and L. Van Gool. Deep retinal image understanding. International Conference on Medical Image Computing and Computer-Assisted Intervention. Berlin: Springer, 2016, pp. 140–148.

  41. Merkow, J., A. Marsden, D. Kriegman, and Z. Tu. Dense volume-to-volume vascular boundary detection. International Conference on Medical Image Computing and Computer-Assisted Intervention. Berlin: Springer, 2016, pp. 371–379.

  42. Michalak, S. M., J. D. Rolston, and M. T. Lawton. Incidence and predictors of complications and mortality in cerebrovascular surgery: National trends from 2007 to 2012. Neurosurgery 79(2):182–193, 2016.

    Article  Google Scholar 

  43. Mo, J., and L. Zhang. Multi-level deep supervised networks for retinal vessel segmentation. Int. J. Comput. Assist. Radiol. Surg. 12(12):2181–2193, 2017.

    Article  Google Scholar 

  44. Moccia, S., E. De Momi, S. El Hadji, and L. S. Mattos. Blood vessel segmentation algorithms – Review of methods, datasets and evaluation metrics. Comput. Methods Progr. Biomed. 158:71–91, 2018.

    Article  Google Scholar 

  45. Moccia, S., G. O. Vanone, E. De Momi, A. Laborai, L. Guastini, G. Peretti, and L. S. Mattos. Learning-based classification of informative laryngoscopic frames. Comput. Methods Progr. Biomed. 158:21–30, 2018.

    Article  Google Scholar 

  46. Morita, A., S. Sora, M. Mitsuishi, S. Warisawa, K. Suruman, D. Asai, J. Arata, S. Baba, H. Takahashi, R. Mochizuki, et al. Microsurgical robotic system for the deep surgical field: development of a prototype and feasibility studies in animal and cadaveric models. J. Neurosurg. 103(2):320–327, 2005.

    Article  Google Scholar 

  47. Motkoski, J. W., and G. R. Sutherland. Why robots entered neurosurgery. Exp. Neurosurg. Anim. Models 116:85–105, 2016.

    Article  Google Scholar 

  48. Muja, M., and D. G. Lowe. Fast approximate nearest neighbors with automatic algorithm configuration. International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications 2(331-340), 2, 2009.

  49. Nah, S., T. H. Kim, and K. M. Lee. Deep multi-scale convolutional neural network for dynamic scene deblurring. IEEE Conference on Computer Vision and Pattern Recognition, 2017.

  50. Niu, P. P., Y. Yu, H. W. Zhou, Y. Liu, Y. Luo, Z. N. Guo, H. Jin, and Y. Yang. Vessel wall differences between middle cerebral artery and basilar artery plaques on magnetic resonance imaging. Sci. Rep. 6:38534 (2016)

  51. Pan, J., L. Zhang, and D. Manocha. Collision-free and smooth trajectory computation in cluttered environments. Int. J. Robot. Res. 31(10):1155–1175, 2012.

    Article  Google Scholar 

  52. Poplin, R., A. V. Varadarajan, K. Blumer, Y. Liu, M. V. McConnell, G. S. Corrado, L. Peng, and D. R. Webster. Prediction of cardiovascular risk factors from retinal fundus photographs via deep learning. Nat. Biomed. Eng. 2:158–164, 2018.

    Article  Google Scholar 

  53. Prentašić, P., M. Heisler, Z. Mammo, S. Lee, A. Merkur, E. Navajas, M. F. Beg, M. Šarunić, and S. Lončarić. Segmentation of the foveal microvasculature using deep learning networks. J. Biomed. Opt. 21(7):075008, 2016.

    Article  Google Scholar 

  54. Prudente, F., S. Moccia, A. Perin, R. Sekula, L. Mattos, J. Balzer, W. Fellows-Mayle, E. De Momi, and C. Riviere. Toward safety enhancement in neurosurgery using a handheld robotic instrument. The Hamlyn Symposium on Medical Robotics, 2017, pp. 15–16.

  55. Ronneberger, O., P. Fischer, and T. Brox. U-net: convolutional networks for biomedical image segmentation. International Conference on Medical Image Computing and Computer-Assisted Intervention. Berlin: Springer, 2015, pp. 234–241.

  56. Salem, N., S. Salem, and A. Nandi. Segmentation of retinal blood vessels based on analysis of the Hessian matrix and clustering algorithm. European Signal Processing Conference, 2007, pp. 428–432.

  57. Sekula, R. F., A. M. Frederickson, G. D. Arnone, M. R. Quigley, and M. Hallett. Microvascular decompression for hemifacial spasm in patients> 65 years of age: an analysis of outcomes and complications. Muscle Nerve 48(5):770–776, 2013.

    Article  Google Scholar 

  58. Smistad, E., and L. Løvstakken. Vessel detection in ultrasound images using deep convolutional neural networks. International Workshop on Large-Scale Annotation of Biomedical Data and Expert Label Synthesis, pp. 30–38. Springer (2016)

  59. Song, J., B. Gonenc, J. Guo, and I. Iordachita. Intraocular snake integrated with the steady-hand eye robot for assisted retinal microsurgery. IEEE International Conference on Robotics and Automation, 2017, pp. 6724–6729.

  60. Sutherland, G. R., P. B. McBeth, and D. F. Louw. Neuroarm: an MR compatible robot for microsurgery. International Congress Series, vol. 1256. Amsterdam: Elsevier, 2003, pp. 504–508.

  61. Sutherland, G. R., S. Wolfsberger, S. Lama, and K. Zareinia. The evolution of neuroArm. Neurosurgery 72(Suppl 1):A27–A32, 2013.

    Article  Google Scholar 

  62. Taylor, R. H., A. Menciassi, G. Fichtinger, P. Fiorini, and P. Dario. Medical robotics and computer-integrated surgery. In: Springer Handbook of Robotics. Cham: Springer, 2016, pp. 1657–1684.

  63. Twinanda, A. P., S. Shehata, D. Mutter, J. Marescaux, M. de Mathelin, and N. Padoy. Endonet: a deep architecture for recognition tasks on laparoscopic videos. IEEE Trans. Med. Imaging 36(1):86–97, 2017.

    Article  Google Scholar 

  64. Wang, S., Y. Yin, G. Cao, B. Wei, Y. Zheng, and G. Yang. Hierarchical retinal blood vessel segmentation based on feature and ensemble learning. Neurocomputing 149:708–717, 2015.

    Article  Google Scholar 

  65. Xue, D. X., R. Zhang, H. Feng, and Y. L. Wang. Cnn-SVM for microvascular morphological type recognition with data augmentation. J. Med. Biol. Eng. 36(6):755–764, 2016.

    Article  Google Scholar 

  66. Yang, S., R. A. MacLachlan, and C. N. Riviere. Design and analysis of 6 DOF handheld micromanipulator. IEEE International Conference on Robotics and Automation, 2012, pp. 1946–1951.

  67. Yang, S., R. A. MacLachlan, and C. N. Riviere. Toward automated intraocular laser surgery using a handheld micromanipulator. IEEE International Conference on Intelligent Robots and Systems, 2014, pp. 1302–1307.

  68. Yang, S., R. A. MacLachlan, and C. N. Riviere. Manipulator design and operation of a six-degree-of-freedom handheld tremor-canceling microsurgical instrument. IEEE/ASME Trans. Mechatron. 20(2):761–772, 2015.

    Article  Google Scholar 

  69. Zhang, B., L. Zhang, L. Zhang, and F. Karray. Retinal vessel extraction by matched filter with first-order derivative of Gaussian. Comput. Biol. Med. 40(4):438–445, 2010.

    Article  Google Scholar 

Download references

Acknowledgments

Partial funding provided by U.S. National Institutes of Health (Grant No. R01EB000526).

Author information

Authors and Affiliations

  1. Department of Advanced Robotics, Istituto Italiano di Tecnologia, Genoa, Italy

    Sara Moccia & Leonardo S. Mattos

  2. Department of Electronics, Information and Bioengineering, Politecnico di Milano, Milan, Italy

    Sara Moccia, Simone Foti, Francesca Prudente & Elena De Momi

  3. Besta NeuroSim Center, IRCCS Istituto Neurologico C. Besta, Milan, Italy

    Alessandro Perin

  4. Department of Neurological Surgery, University of Pittsburgh, Pittsburgh, USA

    Raymond F. Sekula, Jeffrey R. Balzer & Wendy Fellows-Mayle

  5. Robotics Institute, Carnegie Mellon University, Pittsburgh, USA

    Arpita Routray & Cameron N. Riviere

Authors
  1. Sara Moccia
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Simone Foti
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Arpita Routray
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Francesca Prudente
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Alessandro Perin
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Raymond F. Sekula
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Leonardo S. Mattos
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Jeffrey R. Balzer
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Wendy Fellows-Mayle
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Elena De Momi
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Cameron N. Riviere
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Cameron N. Riviere.

Additional information

Associate Editor Kevin Cleary oversaw the review of this article.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Moccia, S., Foti, S., Routray, A. et al. Toward Improving Safety in Neurosurgery with an Active Handheld Instrument. Ann Biomed Eng 46, 1450–1464 (2018). https://doi.org/10.1007/s10439-018-2091-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10439-018-2091-x

Keywords

Search

Navigation