Top

The Journal of Supercomputing

Published in:

15-02-2020

Three-dimensional rapid registration and reconstruction of multi-view rigid objects based on end-to-end deep surface model

Authors: Shengzan Yan, Lijun Xu, Shushan Wang

Published in: The Journal of Supercomputing | Issue 11/2020

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

Three-dimensional object reconstruction from multi-view images is an important topic in computer vision, which has attracted enormous attention during the past decades. With the further study in deep learning, remarkable progress of three-dimensional object reconstruct has been obtained in recent years. In this paper, we proposed three-dimensional rapid registration and reconstruction of multi-view rigid objects based on end-to-end deep surface model in the field of three-dimensional object reconstruction. Firstly, we introduce a matching algorithm called local stereo matching algorithm based on improved census transform and multi-scale spatial, aiming to improve the matching results for those regions. In cost aggregation step, guided map filtering algorithm with excellent gradient preserving property is introduced into Gaussian pyramid structure and regularization is added to strengthen cost volume consistency. Secondly, the improved inception RESNET module is added to improve the feature extraction ability of the network, and multiple features are extracted by using multiple network structures, and finally multiple features are sequentially input into the VRNN module to enhance the reconstruction effect of multi-view images. The experimental results show that our proposed method can not only achieve better reconstruction results, but also reconstruct more details and spend less time in training.

previous article Dynamic power-aware scheduling of real-time tasks for FPGA-based cyber physical systems against power draining hardware trojan attacks

next article Cyber–physical energy system toward safety in smart grids

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Diebel J, Thrun S (2005) An application of Markov random fields to range sensing. In: Advances in Neural Information Processing Systems, vol 24, no 05, pp 291–298

Zhuand J, Yang R (2010) Spatial–temporal fusion for high accuracy depth maps using dynamic MRFs. IEEE Trans Pattern Anal Mach Intell 32(5):899–909

Lu J, Min D, Pahwa RS, Do MN (2011) A review to MRF-based depth map super-resolution and enhancement. In: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp 985–988

Jessop ZM, Al-Sabah A, Gardiner MD, Combellack E, Hawkins K, Whitaker IS (2017) 3D bioprinting for reconstructive surgery: principles, applications and challenges. J Plast Reconstr Aesthet Surg 70(9):1155–1170

Zollhöfer M, Thies J, Garrido P, Bradley D, Beeler T, Pérez P, Stamminger M, Nießner M, Theobalt C (2018) State of the art on monocular 3D face reconstruction, tracking, and applications. Comput Graph Forum 37(2):523–550

Carr R (2013) Coachella picks RBF for 2,200-Acre La Entrada’s infrastructure. Natl Real Estate Invest Exclus Insight 2:67–68

Penczek P, Radermacher M, Frank J (1992) Three-dimensional reconstruction of single particles embedded in ice. Ultramicroscopy 40(1):33–53

Gilbert P (1972) Iterative methods for the three-dimensional reconstruction of an object from projections. J Theor Biol 36(1):105–117

Amenta N, Choi S, Kolluri RK (2001) The power crust, unions of balls, and the medial axis transform. Comput Geom Theory Appl 19(2–3):127–153MathSciNetMATH

10.

Qian P, Jiang Y, Deng Z, Hu L, Sun S, Wang S, Muzic RF (2015) Cluster prototypes and fuzzy memberships jointly leveraged cross-domain maximum entropy clustering. IEEE Trans Cybern 46(1):181–193

11.

Qian P, Jiang Y, Wang S, Su KH, Wang J, Hu L, Muzic RF (2016) Affinity and penalty jointly constrained spectral clustering with all-compatibility, flexibility, and robustness. IEEE Trans Neural Netw Learn Syst 28(5):1123–1138

12.

Qian P, Zhao K, Jiang Y, Su KH, Deng Z, Wang S, Muzic RF (2017) Knowledge-leveraged transfer fuzzy C-means for texture image segmentation with self-adaptive cluster prototype matching. Knowl Based Syst 130:33–50

13.

Qian P, Xi C, Xu M, Jiang Y, Su KH, Wang S, Muzic RF (2018) SSC-EKE: semi-supervised classification with extensive knowledge exploitation. Inf Sci 422:51–76MathSciNetMATH

14.

Qian P, Sun S, Jiang Y, Su KH, Ni T, Wang S, Muzic RF (2016) Cross-domain, soft-partition clustering with diversity measure and knowledge reference. Pattern Recognit 50:155–177

15.

Qian P, Zhou J, Jiang Y, Liang F, Zhao K, Wang S, Su KH, Muzic RF (2018) Multi-view maximum entropy clustering by jointly leveraging inter-view collaborations and intra-view-weighted attributes. IEEE Access 6:28594–28610

16.

Qian P, Chung FL, Wang S, Deng Z (2012) Fast graph-based relaxed clustering for large data sets using minimal enclosing ball. IEEE Trans Syst Man Cybern Part B (Cybern) 42(3):672–687

17.

Jiang Y, Wu D, Deng Z, Qian P, Wang J, Wang G, Chung FL, Choi KS, Wang S (2017) Seizure classification from EEG signals using transfer learning, semi-supervised learning and TSK fuzzy system. IEEE Trans Neural Syst Rehabil Eng 25(12):2270–2284

18.

Jiang Y, Deng Z, Chung FL, Wang G, Qian P, Choi KS, Wang S (2017) Recognition of epileptic EEG signals using a novel multiview TSK fuzzy system. IEEE Trans Fuzzy Syst 25(1):3–20

19.

Jiang Y, Chung FL, Wang S, Deng Z, Wang J, Qian P (2014) Collaborative fuzzy clustering from multiple weighted views. IEEE Trans Cybern 45(4):688–701

20.

Jiang Y, Chung FL, Ishibuchi H, Deng Z, Wang S (2015) Multitask TSK fuzzy system modeling by mining intertask common hidden structure. IEEE Trans Cybern 45(3):534–547

21.

Murphy KP, Weiss Y, Jordan MI (1999) Loopy belief propagation for approximate inference: an empirical study. In: Proceedings of the Fifteenth Conference on Uncertainty in Artificial Intelligence, vol 12, no 9. Morgan Kaufmann Publishers Inc., Burlington, pp 467–475

22.

Park J, Kim H, Tai Y-W, Brown MS, Kweo I (2010) High quality depth map upsampling. In: IEEE International Conference on Computer Vision (ICCV), pp 1623–1630

23.

Yang Q, Yang R, Davis J, Nister D (2007) Spatial-depth super resolution for range images. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 1–8

24.

Chan D, Buisman H et al (2008) A noise-aware filter for real-time depth upsampling. In: Workshop on Multi-camera and Multi-modal Sensor Fusion Algorithms and Applications, pp 209–219

25.

Furukawa Y (2010) Accurate, dense, and robust multiview stereopsis. IEEE Trans Pattern Anal Mach Intell 32(8):1362–1376

26.

Dolson J, Baek J, Plagemann C, Thrun S (2010) Upsampling range data in dynamic environments. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 1141–1148

27.

Foix S, Alenya G, Torras C (2011) Lock-in time-of-flight (TOF) cameras: a survey. Sens J IEEE 11(9):1917–1926

28.

Harrison A, Newman P (2010) Image and sparse laser fusion for dense scene reconstruction. In: Howard A, Iagnemma K, Kelly A (eds) Field and service robotics. Springer, Berlin, pp 219–228

29.

Li N, Gong X, Li H et al (2018) Nonuniform multiview color texture mapping of image sequence and three-dimensional model for faded cultural relics with sift feature points. J Electron Imaging 27(1):1–21MathSciNet

30.

Du C, Du C, Huang L et al (2018) Reconstructing perceived images from human brain activities with Bayesian deep multiview learning. IEEE Trans Neural Netw Learn Syst 24(24):1–14

31.

Yao Y, Luo Z, Li S (2018) MVSNet: depth inference for unstructured multi-view stereo. In: European Conference on Computer Vision

32.

Schoenberg JR, Nathan A, Campbell (2012) Segmentation of dense range information in complex urban scenes. In: 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), vol 21, no 4, pp 2033–2038

33.

Vincent L, Jean Sébastien F, Edmond B (2018) Shape reconstruction using volume sweeping and learned photo consistency. In: European Conference on Computer Vision

34.

Riegler G, Ulusoy AO, Geiger A (2017) OctNet: learning deep 3D representations at high resolutions. In: IEEE Conference on Computer Vision and Pattern Recognition

35.

Riegler G, Ulusoy AO, Bischof H (2017) OctNet fusion: learning depth fusion from data. In: International Conference on 3D Vision

36.

Ji M, Gall J, Zheng H (2017) SurfaceNet: an end-to-end 3D neural network for multiview stereopsis. In: IEEE International Conference on Computer Vision

37.

Zou C, Yumer E, Yang J et al (2017) 3D-PRNN: generating shape primitives with recurrent neural networks, vol 23 no 21, pp 993–1000

38.

Gringarten E, Deutsch CV (2001) Teacher’s aide variogram inter-pretation and modeling. Math Geol 33(4):507–534

39.

Rebecq H, Gallego G, Mueggler E et al (2017) EMVS: event-based multi-view stereo—3D reconstruction with an event camera in real-time. Int J Comput Vis 33(31):980–992

40.

Haopeng Z, Quanmao W, Zhiguo J (2017) 3D reconstruction of space objects from multi-views by a visible sensor. Sensors 17(7):1689–1698

41.

Laviada J, Arboleyaarboleya A, Álvarez Y et al (2017) Multiview three-dimensional reconstruction by millimetre-wave portable camera. Sci Rep 7(1):64–79

42.

Ebner T, Feldmann I, Renault S et al (2017) Multi-view reconstruction of dynamic real-world objects and their integration in augmented and virtual reality applications. J Soc Inf Disp 25(3):151–157

43.

Wang Q, Lv H, Yue J et al (2016) Supervised multiview learning based on simultaneous learning of multiview intact and single view classifier. Neural Comput Appl 16(4):61–73

44.

Sun L, Chen K, Song M et al (2017) Robust, efficient depth reconstruction with hierarchical confidence-based matching. IEEE Trans Image Process 26(7):3331–3343MathSciNetMATH

45.

Huang L, Chao HY, Wang CD (2018) Multi-view intact space clustering. Pattern Recognit 26(7):31–43

46.

Wiles O, Zisserman A (2017) SilNet: single- and multi-view reconstruction by learning from silhouettes. arXiv preprint arXiv:1711.07888

47.

Yin Z, Zheng Y, Doerschuk PC (2001) An ab initio algorithm for low-resolution 3-D reconstructions from cryoelectron microscopy images. J Struct Biol 133(2–3):130–142

48.

Yu L, Fan X, Fa Z et al (2018) DLBI: deep learning guided Bayesian inference for structure reconstruction of super-resolution fluorescence microscopy. Bioinformatics 34(13):284–294

49.

Tang Z, Wang S, Huo J et al (2016) Bayesian framework with non-local and low-rank constraint for image reconstruction. J Phys Conf Ser 787–797:012008

50.

Michelangelo C, Gianvito P, Vladimir K et al (2015) Semi-supervised multi-view learning for gene network reconstruction. PLoS ONE 10(12):31–45

51.

Han X, Gao C, Yu Y (2017) DeepSketch2Face: a deep learning based sketching system for 3D face and caricature modeling. ACM Trans Graph 36(4):1–12

52.

Vodrahalli K, Bhowmik AK (2017) 3D computer vision based on machine learning with deep neural networks: a review. J Soc Inf Disp 25(11):098–103

53.

Raphael P, Mehrdad S, Simon J et al (2018) 3D freehand ultrasound without external tracking using deep learning. Med Image Anal 48:187–202

54.

Zhang J, Li K, Liang Y et al (2017) Learning 3D faces from 2D images via stacked contractive autoencoder. Neurocomputing 257:67–78

55.

Bai S, Zhou Z, Wang J et al (2018) Automatic ensemble diffusion for 3D shape and image retrieval. IEEE Trans Image Process 28(1): 88–101MathSciNetMATH

56.

Hansen MF, Smith ML, Smith LN et al (2018) Automated monitoring of dairy cow body condition, mobility and weight using a single 3D video capture device. Comput Ind 98:14–22

57.

Peng XB, Berseth G, Yin K et al (2017) DeepLoco: dynamic locomotion skills using hierarchical deep reinforcement learning. ACM Trans Graph 36(4):1–13

58.

He GW, Wang TY, Chiang AS et al (2017) Soma detection in 3D images of neurons using machine learning technique. Neuroinformatics 65(45):2081–2099

59.

Zhou W, Yu L, Zhou Y et al (2017) Blind quality estimator for 3D images based on binocular combination and extreme learning machine. Pattern Recognit 71:207–217

Title: Three-dimensional rapid registration and reconstruction of multi-view rigid objects based on end-to-end deep surface model
Authors: Shengzan Yan
Lijun Xu
Shushan Wang
Publication date: 15-02-2020
Publisher: Springer US
Published in: The Journal of Supercomputing / Issue 11/2020
Print ISSN: 0920-8542
Electronic ISSN: 1573-0484
DOI: https://doi.org/10.1007/s11227-020-03194-1

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Springer Professional "Technik"

Springer Professional "Wirtschaft+Technik"

Other articles of this Issue 11/2020

Dynamic power-aware scheduling of real-time tasks for FPGA-based cyber physical systems against power draining hardware trojan attacks

MLEsIDSs: machine learning-based ensembles for intrusion detection systems—a review

Performance modeling of the sparse matrix–vector product via convolutional neural networks

Modified adaptive neuro-fuzzy inference system (M-ANFIS) based multi-disease analysis of healthcare Big Data

A secure biometric-based authentication protocol for global mobility networks in smart cities

A novel LMAEB-CNN model for Chinese microblog sentiment analysis

Premium Partner