Skip to main content
main-content
Top

Hint

Swipe to navigate through the chapters of this book

2021 | OriginalPaper | Chapter

hSDB-instrument: Instrument Localization Database for Laparoscopic and Robotic Surgeries

Authors: Jihun Yoon, Jiwon Lee, Sunghwan Heo, Hayeong Yu, Jayeon Lim, Chi Hyun Song, SeulGi Hong, Seungbum Hong, Bokyung Park, SungHyun Park, Woo Jin Hyung, Min-Kook Choi

Published in: Medical Image Computing and Computer Assisted Intervention – MICCAI 2021

Publisher: Springer International Publishing

share
SHARE

Abstract

Automated surgical instrument localization is an important technology to understand the surgical process and in order to analyze them to provide meaningful guidance during surgery or surgical index after surgery to the surgeon. We introduce a new dataset that reflects the kinematic characteristics of surgical instruments for automated surgical instrument localization of surgical videos. The hSDB(hutom Surgery DataBase)-instrument dataset consists of instrument localization information from 24 cases of laparoscopic cholecystecomy and 24 cases of robotic gastrectomy. Localization information for all instruments is provided in the form of a bounding box for object detection. To handle class imbalance problem between instruments, synthesized instruments modeled in Unity for 3D models are included as training data. Besides, for 3D instrument data, a polygon annotation is provided to enable instance segmentation of the tool. To reflect the kinematic characteristics of all instruments, they are annotated with head and body parts for laparoscopic instruments, and with head, wrist, and body parts for robotic instruments separately. Annotation data of assistive tools (specimen bag, needle, etc.) that are frequently used for surgery are also included. Moreover, we provide statistical information on the hSDB-instrument dataset and the baseline localization performances of the object detection networks trained by the MMDetection library and resulting analyses (The dataset, additional dataset statistics and several trained models are publicly available at https://​hsdb-instrument.​github.​io/​).
Footnotes
1
Additional statistics is described at https://​hsdb-instrument.​github.​io.
 
2
Additional experiment details and results are described at https://​hsdb-instrument.​github.​io/​.
 
Literature
1.
go back to reference Hughes-Hallett, A., Mayer, E.K., Pratt, P.J., Vale, J.A., Darzi, A.W.: Quantitative analysis of technological innovation in minimally invasive surgery. Br. J. Surg. 102(2), 151–157 (2015) CrossRef Hughes-Hallett, A., Mayer, E.K., Pratt, P.J., Vale, J.A., Darzi, A.W.: Quantitative analysis of technological innovation in minimally invasive surgery. Br. J. Surg. 102(2), 151–157 (2015) CrossRef
2.
go back to reference Perez, R.E., Schwaitzberg, S.D.: Robotic surgery: finding value in 2019 and beyond. Ann. Laparosc. Endosc. Surg. 4(51) (2019) Perez, R.E., Schwaitzberg, S.D.: Robotic surgery: finding value in 2019 and beyond. Ann. Laparosc. Endosc. Surg. 4(51) (2019)
3.
go back to reference Jin, A., et al.: Tool detection and operative skill assessment in surgical videos using region-based convolutional neural networks. In: Proceedings of WACV (2018) Jin, A., et al.: Tool detection and operative skill assessment in surgical videos using region-based convolutional neural networks. In: Proceedings of WACV (2018)
4.
go back to reference Maier-Hein, L., Wagner, M., Ross, T., et al.: Heidelberg colorectal data set for surgical data science in the sensor operating room. Sci. Data 8, 101 (2021) CrossRef Maier-Hein, L., Wagner, M., Ross, T., et al.: Heidelberg colorectal data set for surgical data science in the sensor operating room. Sci. Data 8, 101 (2021) CrossRef
5.
go back to reference Twinanda, A.P., Shehata, S., Mutter, D., Marescaux, J., de Mathelin, M., Padoy, N.: EndoNet: a deep architecture for recognition tasks on laparoscopic videos. IEEE Trans. Med. Imaging 36(1), 86–97 (2017) CrossRef Twinanda, A.P., Shehata, S., Mutter, D., Marescaux, J., de Mathelin, M., Padoy, N.: EndoNet: a deep architecture for recognition tasks on laparoscopic videos. IEEE Trans. Med. Imaging 36(1), 86–97 (2017) CrossRef
7.
go back to reference Sarikaya, D., Corso, J.J., Guru, K.A.: Detection and localization of robotic tools in robot-assisted surgery videos using deep neural networks for region proposal and detection. IEEE Trans. Med. Imaging 36(7), 1542–1549 (2017) CrossRef Sarikaya, D., Corso, J.J., Guru, K.A.: Detection and localization of robotic tools in robot-assisted surgery videos using deep neural networks for region proposal and detection. IEEE Trans. Med. Imaging 36(7), 1542–1549 (2017) CrossRef
9.
go back to reference Geiger, A., Lenz, P., Urtasun, R.: Are we ready for autonomous driving? The KITTI vision benchmark suite. In: Proceedings of CVPR (2012) Geiger, A., Lenz, P., Urtasun, R.: Are we ready for autonomous driving? The KITTI vision benchmark suite. In: Proceedings of CVPR (2012)
10.
go back to reference Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. In: Proceedings of NIPS (2015) Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. In: Proceedings of NIPS (2015)
12.
go back to reference Lin, T.-Y., Dolláir, P., Girshick, R., He, K., Hariharan, B., Belongie, S.: Feature pyramid networks for object detection. In: Proceedings of CVPR (2017) Lin, T.-Y., Dolláir, P., Girshick, R., He, K., Hariharan, B., Belongie, S.: Feature pyramid networks for object detection. In: Proceedings of CVPR (2017)
13.
go back to reference Huang, X., Liu, M.-Y., Belongie, S., Kautz J.: Multimodal unsupervised image-to-image translation. In: Proceedings of ECCV (2018) Huang, X., Liu, M.-Y., Belongie, S., Kautz J.: Multimodal unsupervised image-to-image translation. In: Proceedings of ECCV (2018)
14.
go back to reference Lee, K., Choi, M. -K., Jung, H.: DavinciGAN: unpaired surgical instrument translation for data augmentation. In: Proceedings of MIDL (2019) Lee, K., Choi, M. -K., Jung, H.: DavinciGAN: unpaired surgical instrument translation for data augmentation. In: Proceedings of MIDL (2019)
15.
go back to reference Park, T., Liu, M.-Y., Wang, T.-C., Zhu, J.-Y.: Semantic image synthesis with spatially-adaptive normalization. In: Proceedings of CVPR (2019) Park, T., Liu, M.-Y., Wang, T.-C., Zhu, J.-Y.: Semantic image synthesis with spatially-adaptive normalization. In: Proceedings of CVPR (2019)
18.
go back to reference Paszke, A., et al.: PyTorch: an imperative style, high-performance deep learning library. In: Proceedings of NeurIPS (2019) Paszke, A., et al.: PyTorch: an imperative style, high-performance deep learning library. In: Proceedings of NeurIPS (2019)
20.
go back to reference Tremblay, J., et al.: Training deep networks with synthetic data: bridging the reality gap by domain randomization. In: Proceedings of CVPRW (2018) Tremblay, J., et al.: Training deep networks with synthetic data: bridging the reality gap by domain randomization. In: Proceedings of CVPRW (2018)
21.
go back to reference Cai Z., Vasconcelos, N.: Cascade R-CNN: delving into high quality object detection. In: Proceedings of CVPR (2018) Cai Z., Vasconcelos, N.: Cascade R-CNN: delving into high quality object detection. In: Proceedings of CVPR (2018)
22.
go back to reference Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollar, P.: Focal loss for dense object detection. In: Proceedings of ICCV (2017) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollar, P.: Focal loss for dense object detection. In: Proceedings of ICCV (2017)
23.
go back to reference Tian, Z., Shen, C., Chen, H., He, T.: FCOS: fully convolutional one-stage object detection. In: Proceedings of ICCV (2019) Tian, Z., Shen, C., Chen, H., He, T.: FCOS: fully convolutional one-stage object detection. In: Proceedings of ICCV (2019)
25.
go back to reference Pang, J., Chen, K., Shi, J., Feng, H., Ouyang, W., Lin, D.: Libra R-CNN: towards balanced learning for object detection. In: Proceedings of CVPR (2019) Pang, J., Chen, K., Shi, J., Feng, H., Ouyang, W., Lin, D.: Libra R-CNN: towards balanced learning for object detection. In: Proceedings of CVPR (2019)
26.
go back to reference Li, A., Yang, X., Zhang, C.: Rethinking classification and localization for object detection. In: Proceedings of BMVC (2019) Li, A., Yang, X., Zhang, C.: Rethinking classification and localization for object detection. In: Proceedings of BMVC (2019)
27.
go back to reference He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of CVPR (2016) He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of CVPR (2016)
28.
go back to reference Xie, S., Girshick, R., Dolláir, P., Tu, Z., He K.: Aggregated residual transformations for deep neural networks. In: Proceedings of CVPR (2017) Xie, S., Girshick, R., Dolláir, P., Tu, Z., He K.: Aggregated residual transformations for deep neural networks. In: Proceedings of CVPR (2017)
Metadata
Title
hSDB-instrument: Instrument Localization Database for Laparoscopic and Robotic Surgeries
Authors
Jihun Yoon
Jiwon Lee
Sunghwan Heo
Hayeong Yu
Jayeon Lim
Chi Hyun Song
SeulGi Hong
Seungbum Hong
Bokyung Park
SungHyun Park
Woo Jin Hyung
Min-Kook Choi
Copyright Year
2021
DOI
https://doi.org/10.1007/978-3-030-87202-1_38

Premium Partner