Skip to main content
Top
Published in: Neural Computing and Applications 13/2021

06-04-2021 | S.I. : DICTA 2019

Fixed-Lens camera setup and calibrated image registration for multifocus multiview 3D reconstruction

Authors: Shah Ariful Hoque Chowdhury, Chuong Nguyen, Hengjia Li, Richard Hartley

Published in: Neural Computing and Applications | Issue 13/2021

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Image-based 3D reconstruction or 3D photogrammetry of small-scale objects including insects and biological specimens is challenging due to the use of a high magnification lens with inherently limited depth of field, and the object’s fine structures. Therefore, the traditional 3D reconstruction techniques cannot be applied without additional image preprocessing. One such preprocessing technique is multifocus stacking/fusion that combines a set of partially focused images captured at different distances from the same viewing angle to create a single in-focus image. We found that the image formation is not properly considered by the traditional multifocus image capture and stacking techniques. The resulting in-focus images contain artifacts that violate the perspective projection. A 3D reconstruction using such images often fails to produce accurate 3D models of the captured objects. This paper shows how this problem can be solved effectively by a new multifocus multiview 3D reconstruction procedure which includes a new Fixed-Lens multifocus image capture and a calibrated image registration technique using analytic homography transformation. The experimental results using the real and synthetic images demonstrate the effectiveness of the proposed solutions by showing that both the fixed-lens image capture and multifocus stacking with calibrated image alignment significantly reduce the errors in the camera poses and produce more complete 3D reconstructed models as compared with those by the conventional moving lens image capture and multifocus stacking.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Appendix
Available only for authorised users
Literature
4.
go back to reference Amin-Naji M, Aghagolzadeh A, Ezoji M (2018) Fully Convolutional Networks for Multi-Focus Image Fusion. In: 2018 9th International Symposium on Telecommunications (IST), pp 553–558 Amin-Naji M, Aghagolzadeh A, Ezoji M (2018) Fully Convolutional Networks for Multi-Focus Image Fusion. In: 2018 9th International Symposium on Telecommunications (IST), pp 553–558
9.
go back to reference Burt P, Adelson E (1983) The Laplacian Pyramid as a Compact Image Code. IEEE Trans Commun 31(4):532–540CrossRef Burt P, Adelson E (1983) The Laplacian Pyramid as a Compact Image Code. IEEE Trans Commun 31(4):532–540CrossRef
10.
go back to reference Cignoni P, Callieri M, Corsini M, Dellepiane M, Ganovelli F, Ranzuglia G (2008) MeshLab: an Open-Source Mesh Processing Tool. In: Eurographics Italian Chapter Conference Cignoni P, Callieri M, Corsini M, Dellepiane M, Ganovelli F, Ranzuglia G (2008) MeshLab: an Open-Source Mesh Processing Tool. In: Eurographics Italian Chapter Conference
14.
go back to reference Fujii H, Kodama K, Hamamoto T (2016) Scene flow estimation through 3D analysis of multi-focus images. In: 2016 Visual Communications and Image Processing (VCIP), IEEE, pp 1–4 Fujii H, Kodama K, Hamamoto T (2016) Scene flow estimation through 3D analysis of multi-focus images. In: 2016 Visual Communications and Image Processing (VCIP), IEEE, pp 1–4
17.
go back to reference Guo X, Nie R, Cao J, Zhou D, Qian W (2018) Fully Convolutional Network-Based Multifocus Image Fusion. Neural Comput 30(7):1775–1800MathSciNetCrossRef Guo X, Nie R, Cao J, Zhou D, Qian W (2018) Fully Convolutional Network-Based Multifocus Image Fusion. Neural Comput 30(7):1775–1800MathSciNetCrossRef
18.
go back to reference Hartley R, Zisserman A (2003) Multiple View Geometry in Computer Vision, 2nd edn. Cambridge University Press, USAMATH Hartley R, Zisserman A (2003) Multiple View Geometry in Computer Vision, 2nd edn. Cambridge University Press, USAMATH
19.
go back to reference Hecht E (2002) Optics. Addison Wesley Longman, Reading, Massachusetts Hecht E (2002) Optics. Addison Wesley Longman, Reading, Massachusetts
21.
go back to reference Hui Li, Manjunath BS, Mitra SK (1994) Multi-sensor image fusion using the wavelet transform. In: Proceedings of 1st International Conference on Image Processing, vol 1, pp 51–55 vol.1 Hui Li, Manjunath BS, Mitra SK (1994) Multi-sensor image fusion using the wavelet transform. In: Proceedings of 1st International Conference on Image Processing, vol 1, pp 51–55 vol.1
22.
go back to reference Ji Z, Kang X, Zhang K, Duan P, Hao Q (2019) A Two-Stage Multi-Focus Image Fusion Framework Robust to Image Mis-Registration. IEEE Access 7:123231–123243CrossRef Ji Z, Kang X, Zhang K, Duan P, Hao Q (2019) A Two-Stage Multi-Focus Image Fusion Framework Robust to Image Mis-Registration. IEEE Access 7:123231–123243CrossRef
24.
go back to reference Kodama K, Wang Z, Sato M, Murakami T (2017) Real-time 3-D image reconstruction from multi-focus images by efficient linear filtering with multi-dimensional symmetry. In: 2017 IEEE International Conference on Image Processing (ICIP), pp 3575–3579, https://doi.org/10.1109/ICIP.2017.8296948 Kodama K, Wang Z, Sato M, Murakami T (2017) Real-time 3-D image reconstruction from multi-focus images by efficient linear filtering with multi-dimensional symmetry. In: 2017 IEEE International Conference on Image Processing (ICIP), pp 3575–3579, https://​doi.​org/​10.​1109/​ICIP.​2017.​8296948
26.
go back to reference Li H, Nguyen C (2019) Perspective-consistent multifocus multiview 3D reconstruction of small objects. In: 2019 Digital Image Computing: Techniques and Applications (DICTA), IEEE, pp 1–8 Li H, Nguyen C (2019) Perspective-consistent multifocus multiview 3D reconstruction of small objects. In: 2019 Digital Image Computing: Techniques and Applications (DICTA), IEEE, pp 1–8
28.
go back to reference Li S, Kang X, Hu J (2013) Image Fusion With Guided Filtering. IEEE Trans Image Proc 22(7):2864–2875CrossRef Li S, Kang X, Hu J (2013) Image Fusion With Guided Filtering. IEEE Trans Image Proc 22(7):2864–2875CrossRef
30.
go back to reference Liang Y, Mao Y, Tang Z, Yan M, Zhao Y, Liu J (2019) Efficient misalignment-robust multi-focus microscopical images fusion. Sign Proc 161:111–123CrossRef Liang Y, Mao Y, Tang Z, Yan M, Zhao Y, Liu J (2019) Efficient misalignment-robust multi-focus microscopical images fusion. Sign Proc 161:111–123CrossRef
35.
go back to reference Mishra D, Palkar B (2015) Image Fusion Techniques: A Review. Int J Comput Appl 130:7–13 Mishra D, Palkar B (2015) Image Fusion Techniques: A Review. Int J Comput Appl 130:7–13
39.
go back to reference Nguyen CV, Lovell DR, Adcock M, La Salle J (2014) Capturing natural-colour 3D models of insects for species discovery and diagnostics. PloS one 9(4):e94346CrossRef Nguyen CV, Lovell DR, Adcock M, La Salle J (2014) Capturing natural-colour 3D models of insects for species discovery and diagnostics. PloS one 9(4):e94346CrossRef
43.
go back to reference Sakamoto T, Kodama K, Hamamoto T (2012a) A novel scheme for 4-D light-field compression based on 3-D representation by multi-focus images. In: 2012 19th IEEE international conference on image processing, IEEE, pp 2901–2904 Sakamoto T, Kodama K, Hamamoto T (2012a) A novel scheme for 4-D light-field compression based on 3-D representation by multi-focus images. In: 2012 19th IEEE international conference on image processing, IEEE, pp 2901–2904
44.
go back to reference Sakamoto T, Kodama K, Hamamoto T (2012b) A study on efficient compression of multi-focus images for dense light-field reconstruction. In: 2012 Visual Communications and Image Processing, IEEE, pp 1–6 Sakamoto T, Kodama K, Hamamoto T (2012b) A study on efficient compression of multi-focus images for dense light-field reconstruction. In: 2012 Visual Communications and Image Processing, IEEE, pp 1–6
45.
go back to reference Schönberger JL, contributors (2020) COLMAP: a general-purpose Structure-from-Motion (SfM) and Multi-View Stereo (MVS) pipeline. https://github.com/colmap, [Online; accessed 15-November-2020] Schönberger JL, contributors (2020) COLMAP: a general-purpose Structure-from-Motion (SfM) and Multi-View Stereo (MVS) pipeline. https://​github.​com/​colmap, [Online; accessed 15-November-2020]
46.
go back to reference Seitz SM, Curless B, Diebel J, Scharstein D, Szeliski R (2006) A comparison and evaluation of multi-view stereo reconstruction algorithms. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06), IEEE, vol 1, pp 519–528 Seitz SM, Curless B, Diebel J, Scharstein D, Szeliski R (2006) A comparison and evaluation of multi-view stereo reconstruction algorithms. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06), IEEE, vol 1, pp 519–528
49.
go back to reference Ströbel B, Schmelzle S, Blüthgen N, Heethoff M (2018) An automated device for the digitization and 3D modelling of insects, combining extended-depth-of-field and all-side multi-view imaging. ZooKeys 759:1CrossRef Ströbel B, Schmelzle S, Blüthgen N, Heethoff M (2018) An automated device for the digitization and 3D modelling of insects, combining extended-depth-of-field and all-side multi-view imaging. ZooKeys 759:1CrossRef
52.
go back to reference Szeliski R (2010) Comput Vision: Algorithms Appl, 1st edn. Springer-Verlag, Berlin, Heidelberg Szeliski R (2010) Comput Vision: Algorithms Appl, 1st edn. Springer-Verlag, Berlin, Heidelberg
55.
go back to reference Xu H, Fan F, Zhang H, Le Z, Huang J (2020) A Deep Model for Multi-Focus Image Fusion Based on Gradients and Connected Regions. IEEE Access 8:26316–26327CrossRef Xu H, Fan F, Zhang H, Le Z, Huang J (2020) A Deep Model for Multi-Focus Image Fusion Based on Gradients and Connected Regions. IEEE Access 8:26316–26327CrossRef
56.
go back to reference Yang B, Li S (2010) Multifocus Image Fusion and Restoration With Sparse Representation. IEEE Trans Instrum Measure 59(4):884–892CrossRef Yang B, Li S (2010) Multifocus Image Fusion and Restoration With Sparse Representation. IEEE Trans Instrum Measure 59(4):884–892CrossRef
62.
go back to reference Zhu D, Wu C, Liu Y, Fu D (2018) 3D reconstruction based on light field images. In: Yu H, Dong J (eds) Ninth International Conference on Graphic and Image Processing (ICGIP 2017), International Society for Optics and Photonics, SPIE, vol 10615, pp 951 – 959, https://doi.org/10.1117/12.2304504, Zhu D, Wu C, Liu Y, Fu D (2018) 3D reconstruction based on light field images. In: Yu H, Dong J (eds) Ninth International Conference on Graphic and Image Processing (ICGIP 2017), International Society for Optics and Photonics, SPIE, vol 10615, pp 951 – 959, https://​doi.​org/​10.​1117/​12.​2304504,
63.
go back to reference Zitova B, Flusser J (2003) Image registration methods: a survey. Image Vision Comput 21(11):977–1000CrossRef Zitova B, Flusser J (2003) Image registration methods: a survey. Image Vision Comput 21(11):977–1000CrossRef
Metadata
Title
Fixed-Lens camera setup and calibrated image registration for multifocus multiview 3D reconstruction
Authors
Shah Ariful Hoque Chowdhury
Chuong Nguyen
Hengjia Li
Richard Hartley
Publication date
06-04-2021
Publisher
Springer London
Published in
Neural Computing and Applications / Issue 13/2021
Print ISSN: 0941-0643
Electronic ISSN: 1433-3058
DOI
https://doi.org/10.1007/s00521-021-05926-7

Other articles of this Issue 13/2021

Neural Computing and Applications 13/2021 Go to the issue

Premium Partner