Skip to main content
Erschienen in: Neural Processing Letters 2/2020

08.01.2020

3D Model Retrieval Using Bipartite Graph Matching Based on Attention

verfasst von: Shanlin Sun, Yun Li, Yunfeng Xie, Zhicheng Tan, Xing Yao, Rongyao Zhang

Erschienen in: Neural Processing Letters | Ausgabe 2/2020

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

In this paper, we propose an attention-based bipartite graph 3D model retrieval algorithm, where many-to-many matching method, the weighted bipartite graph matching, is employed for comparison between two 3D models. Considering the panoramic views can donate the spatial and structural information, in this work, we use panoramic views to represent each 3D model. Attention mechanism is used to generate the weight of all views of each model. And then, we construct a weighted bipartite graph with the views of those models and the weight of each view. According to the bipartite graph, the matching result is used to measure the similarity between two 3D models. We experiment our method on ModelNet, NTU and ETH datasets, and the experimental results and comparison with other methods show the effectiveness of our method.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Ansary TF, Daoudi M, Vandeborre J-P (2007) A bayesian 3-d search engine using adaptive views clustering. IEEE Trans Multimed 9(1):78–88CrossRef Ansary TF, Daoudi M, Vandeborre J-P (2007) A bayesian 3-d search engine using adaptive views clustering. IEEE Trans Multimed 9(1):78–88CrossRef
2.
Zurück zum Zitat Bahdanau D, Cho K, Bengio Y (2014) Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473 Bahdanau D, Cho K, Bengio Y (2014) Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:​1409.​0473
3.
Zurück zum Zitat Bai S, Bai X, Zhou Z, Zhang Z, Jan Latecki L (2016) Gift: a real-time and scalable 3d shape search engine. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5023–5032 Bai S, Bai X, Zhou Z, Zhang Z, Jan Latecki L (2016) Gift: a real-time and scalable 3d shape search engine. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5023–5032
4.
Zurück zum Zitat Bimbo AD, Pala P (2006) Content-based retrieval of 3d models. ACM Trans Multimed Comput Commun Appl (TOMM) 2(1):20–43CrossRef Bimbo AD, Pala P (2006) Content-based retrieval of 3d models. ACM Trans Multimed Comput Commun Appl (TOMM) 2(1):20–43CrossRef
5.
Zurück zum Zitat Chen DY, Tian XP, Shen YT, Ming O (2003) On visual similarity based 3d model retrieval. Comput Graph Forum 22(3):223–232CrossRef Chen DY, Tian XP, Shen YT, Ming O (2003) On visual similarity based 3d model retrieval. Comput Graph Forum 22(3):223–232CrossRef
6.
Zurück zum Zitat Chen J, Zhang H, He X, Nie L, Liu W, Chua T-S (2017) Attentive collaborative filtering: multimedia recommendation with item-and component-level attention. In: Proceedings of the 40th international ACM SIGIR conference on research and development in information retrieval, pp 335–344. ACM Chen J, Zhang H, He X, Nie L, Liu W, Chua T-S (2017) Attentive collaborative filtering: multimedia recommendation with item-and component-level attention. In: Proceedings of the 40th international ACM SIGIR conference on research and development in information retrieval, pp 335–344. ACM
7.
Zurück zum Zitat Chiotellis I, Triebel R, Windheuser T, Cremers D (2016) Non-rigid 3d shape retrieval via large margin nearest neighbor embedding. In: European conference on computer vision, pp 327–342. Springer Chiotellis I, Triebel R, Windheuser T, Cremers D (2016) Non-rigid 3d shape retrieval via large margin nearest neighbor embedding. In: European conference on computer vision, pp 327–342. Springer
8.
Zurück zum Zitat Daras P, Axenopoulos A (2009) A compact multi-view descriptor for 3d object retrieval. In: International workshop on content-based multimedia indexing, pp 115–119 Daras P, Axenopoulos A (2009) A compact multi-view descriptor for 3d object retrieval. In: International workshop on content-based multimedia indexing, pp 115–119
9.
Zurück zum Zitat Feng Y, Zhang Z, Zhao X, Ji R, Gao Y (2018) GVCNN: Group-view convolutional neural networks for 3d shape recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 264–272 Feng Y, Zhang Z, Zhao X, Ji R, Gao Y (2018) GVCNN: Group-view convolutional neural networks for 3d shape recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 264–272
10.
Zurück zum Zitat Gao Y, Tang J, Hong R, Yan S, Dai Q, Zhang N, Chua T-S (2012) Camera constraint-free view-based 3-d object retrieval. IEEE Trans Image Process 21(4):2269–2281MathSciNetCrossRef Gao Y, Tang J, Hong R, Yan S, Dai Q, Zhang N, Chua T-S (2012) Camera constraint-free view-based 3-d object retrieval. IEEE Trans Image Process 21(4):2269–2281MathSciNetCrossRef
11.
Zurück zum Zitat Gao Y, Wang M, Tao D, Ji R, Dai Q (2012) 3-d object retrieval and recognition with hypergraph analysis. IEEE Trans Image Process 21(9):4290–4303MathSciNetCrossRef Gao Y, Wang M, Tao D, Ji R, Dai Q (2012) 3-d object retrieval and recognition with hypergraph analysis. IEEE Trans Image Process 21(9):4290–4303MathSciNetCrossRef
12.
Zurück zum Zitat Garcia-Garcia A, Gomez-Donoso F, Garcia-Rodriguez J, Orts-Escolano S, Cazorla M, Azorin-Lopez J (2016) Pointnet: a 3d convolutional neural network for real-time object class recognition. In: 2016 International joint conference on neural networks (IJCNN), pp 1578–1584. IEEE Garcia-Garcia A, Gomez-Donoso F, Garcia-Rodriguez J, Orts-Escolano S, Cazorla M, Azorin-Lopez J (2016) Pointnet: a 3d convolutional neural network for real-time object class recognition. In: 2016 International joint conference on neural networks (IJCNN), pp 1578–1584. IEEE
13.
Zurück zum Zitat Guo H, Wang J, Gao Y, Li J, Lu H (2016) Multi-view 3d object retrieval with deep embedding network. IEEE Trans Image Process Publ IEEE Signal Process Soc 25(12):5526–5537MathSciNetCrossRef Guo H, Wang J, Gao Y, Li J, Lu H (2016) Multi-view 3d object retrieval with deep embedding network. IEEE Trans Image Process Publ IEEE Signal Process Soc 25(12):5526–5537MathSciNetCrossRef
14.
Zurück zum Zitat He X, Chua T-S (2017) Neural factorization machines for sparse predictive analytics. In: Proceedings of the 40th international ACM SIGIR conference on research and development in information retrieval, pp 355–364. ACM He X, Chua T-S (2017) Neural factorization machines for sparse predictive analytics. In: Proceedings of the 40th international ACM SIGIR conference on research and development in information retrieval, pp 355–364. ACM
15.
Zurück zum Zitat Hilaga M, Shinagawa Y, Kohmura T, Kunii TL (2001) Topology matching for fully automatic similarity estimation of 3d shapes. In: Proceedings of the 28th annual conference on computer graphics and interactive techniques, pp 203–212. ACM Hilaga M, Shinagawa Y, Kohmura T, Kunii TL (2001) Topology matching for fully automatic similarity estimation of 3d shapes. In: Proceedings of the 28th annual conference on computer graphics and interactive techniques, pp 203–212. ACM
16.
Zurück zum Zitat Kanezaki A, Matsushita Y, Nishida Y (2016) Rotationnet: joint object categorization and pose estimation using multiviews from unsupervised viewpoints. arXiv preprint arXiv:1603.06208 Kanezaki A, Matsushita Y, Nishida Y (2016) Rotationnet: joint object categorization and pose estimation using multiviews from unsupervised viewpoints. arXiv preprint arXiv:​1603.​06208
17.
Zurück zum Zitat Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105 Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105
18.
Zurück zum Zitat Leibe B, Schiele B (2003) Analyzing appearance and contour based methods for object categorization. In: 2003 IEEE Computer society conference on computer vision and pattern recognition, 2003. Proceedings, vol 2, pp II–409. IEEE Leibe B, Schiele B (2003) Analyzing appearance and contour based methods for object categorization. In: 2003 IEEE Computer society conference on computer vision and pattern recognition, 2003. Proceedings, vol 2, pp II–409. IEEE
19.
Zurück zum Zitat Leng B, Guo S, Changchun D, Zeng J, Xiong Z (2017) 3d object retrieval based on viewpoint segmentation. Multimed Syst 23(1):19–28CrossRef Leng B, Guo S, Changchun D, Zeng J, Xiong Z (2017) 3d object retrieval based on viewpoint segmentation. Multimed Syst 23(1):19–28CrossRef
20.
Zurück zum Zitat Liu A, Wang Z, Nie W, Yuting S (2015) Graph-based characteristic view set extraction and matching for 3d model retrieval. Inf Sci 320:429–442CrossRef Liu A, Wang Z, Nie W, Yuting S (2015) Graph-based characteristic view set extraction and matching for 3d model retrieval. Inf Sci 320:429–442CrossRef
21.
Zurück zum Zitat Liu A-A, Nie W-Z, Gao Y, Yu-Ting S (2018) View-based 3-d model retrieval: a benchmark. IEEE Trans Cybern 48(3):916–928 Liu A-A, Nie W-Z, Gao Y, Yu-Ting S (2018) View-based 3-d model retrieval: a benchmark. IEEE Trans Cybern 48(3):916–928
22.
Zurück zum Zitat Liu A-A, Nie W, Su Y (2019) 3d object retrieval based on multi-view latent variable model. IEEE Trans Circuits Syst Video Technol 29(3):868–880CrossRef Liu A-A, Nie W, Su Y (2019) 3d object retrieval based on multi-view latent variable model. IEEE Trans Circuits Syst Video Technol 29(3):868–880CrossRef
23.
Zurück zum Zitat Maturana D, Scherer S (2015) Voxnet: a 3d convolutional neural network for real-time object recognition. In: 2015 IEEE/RSJ International conference on intelligent robots and systems (IROS), pp 922–928. IEEE Maturana D, Scherer S (2015) Voxnet: a 3d convolutional neural network for real-time object recognition. In: 2015 IEEE/RSJ International conference on intelligent robots and systems (IROS), pp 922–928. IEEE
24.
Zurück zum Zitat Osada R, Funkhouser T, Chazelle B, Dobkin D (2002) Shape distributions. ACM Trans Graph (TOG) 21(4):807–832MathSciNetCrossRef Osada R, Funkhouser T, Chazelle B, Dobkin D (2002) Shape distributions. ACM Trans Graph (TOG) 21(4):807–832MathSciNetCrossRef
25.
Zurück zum Zitat Papadakis P, Pratikakis I, Theoharis T, Perantonis S (2010) Panorama: a 3d shape descriptor based on panoramic views for unsupervised 3d object retrieval. Int J Comput Vis 89(2–3):177–192CrossRef Papadakis P, Pratikakis I, Theoharis T, Perantonis S (2010) Panorama: a 3d shape descriptor based on panoramic views for unsupervised 3d object retrieval. Int J Comput Vis 89(2–3):177–192CrossRef
26.
Zurück zum Zitat Sfikas K, Pratikakis I, Theoharis T (2018) Ensemble of panorama-based convolutional neural networks for 3d model classification and retrieval. Comput Graph 71:208–218CrossRef Sfikas K, Pratikakis I, Theoharis T (2018) Ensemble of panorama-based convolutional neural networks for 3d model classification and retrieval. Comput Graph 71:208–218CrossRef
27.
Zurück zum Zitat Sfikas K, Theoharis T, Pratikakis I (2017) Exploiting the panorama representation for convolutional neural network classification and retrieval. In: Eurographics workshop on 3D object retrieval, vol 8. The Eurographics Association Sfikas K, Theoharis T, Pratikakis I (2017) Exploiting the panorama representation for convolutional neural network classification and retrieval. In: Eurographics workshop on 3D object retrieval, vol 8. The Eurographics Association
28.
Zurück zum Zitat Shi B, Bai S, Zhou Z, Bai X (2015) Deeppano: deep panoramic representation for 3-d shape recognition. IEEE Signal Process Lett 22(12):2339–2343CrossRef Shi B, Bai S, Zhou Z, Bai X (2015) Deeppano: deep panoramic representation for 3-d shape recognition. IEEE Signal Process Lett 22(12):2339–2343CrossRef
29.
Zurück zum Zitat Shu Z, Xin S, Huixia X, Kavan L, Wang P, Liu L (2016) 3d model classification via principal thickness images. Comput Aided Des 78:199–208CrossRef Shu Z, Xin S, Huixia X, Kavan L, Wang P, Liu L (2016) 3d model classification via principal thickness images. Comput Aided Des 78:199–208CrossRef
30.
Zurück zum Zitat Su H, Maji S, Kalogerakis E, Learned-Miller E (2015) Multi-view convolutional neural networks for 3d shape recognition. In: Proceedings of the IEEE international conference on computer vision, pp 945–953 Su H, Maji S, Kalogerakis E, Learned-Miller E (2015) Multi-view convolutional neural networks for 3d shape recognition. In: Proceedings of the IEEE international conference on computer vision, pp 945–953
31.
Zurück zum Zitat Su H, Maji S, Kalogerakis E, Learned-Miller EG (2015) Multi-view convolutional neural networks for 3d shape recognition. ICCV, pp 945–953 Su H, Maji S, Kalogerakis E, Learned-Miller EG (2015) Multi-view convolutional neural networks for 3d shape recognition. ICCV, pp 945–953
32.
Zurück zum Zitat Vranic DV (2003) An improvement of rotation invariant 3d-shape based on functions on concentric spheres. In: 2003 International conference on image processing, 2003. ICIP 2003. Proceedings, vol 3, pp III–757. IEEE Vranic DV (2003) An improvement of rotation invariant 3d-shape based on functions on concentric spheres. In: 2003 International conference on image processing, 2003. ICIP 2003. Proceedings, vol 3, pp III–757. IEEE
33.
Zurück zum Zitat Wang D, Wang B, Zhao S, Yao H, Liu H (2017) View-based 3d object retrieval with discriminative views. Neurocomputing 252(C):58–66CrossRef Wang D, Wang B, Zhao S, Yao H, Liu H (2017) View-based 3d object retrieval with discriminative views. Neurocomputing 252(C):58–66CrossRef
34.
Zurück zum Zitat Nie W, Liu A-A, Gao Y, Su Y (2019) Hyper-clique graph matching and applications. IEEE Trans Circuits Syst Video Technol 29(6):1619–1630CrossRef Nie W, Liu A-A, Gao Y, Su Y (2019) Hyper-clique graph matching and applications. IEEE Trans Circuits Syst Video Technol 29(6):1619–1630CrossRef
35.
Zurück zum Zitat Wenshan H, Liu G-P, Zhou H (2013) Web-based 3-d control laboratory for remote real-time experimentation. IEEE Trans Ind Electron 60(10):4673–4682CrossRef Wenshan H, Liu G-P, Zhou H (2013) Web-based 3-d control laboratory for remote real-time experimentation. IEEE Trans Ind Electron 60(10):4673–4682CrossRef
36.
Zurück zum Zitat Wu Z, Song S, Khosla A, Yu F, Zhang L, Tang X, Xiao J (2015) 3d shapenets: a deep representation for volumetric shapes. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1912–1920 Wu Z, Song S, Khosla A, Yu F, Zhang L, Tang X, Xiao J (2015) 3d shapenets: a deep representation for volumetric shapes. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1912–1920
37.
Zurück zum Zitat Zhang H, Kyaw Z, Chang S-F, Chua T-S (2017) Visual translation embedding network for visual relation detection. In: CVPR, vol 1, p 5 Zhang H, Kyaw Z, Chang S-F, Chua T-S (2017) Visual translation embedding network for visual relation detection. In: CVPR, vol 1, p 5
38.
Zurück zum Zitat Zhang H, Kyaw Z, Yu J, Chang S-F (2017) PPR-FCN: Weakly supervised visual relation detection via parallel pairwise R-FCN. arXiv preprint arXiv:1708.01956 Zhang H, Kyaw Z, Yu J, Chang S-F (2017) PPR-FCN: Weakly supervised visual relation detection via parallel pairwise R-FCN. arXiv preprint arXiv:​1708.​01956
39.
Zurück zum Zitat Zhang H, Niu Y, Chang S-F (2018) Grounding referring expressions in images by variational context. In: The IEEE conference on computer vision and pattern recognition Zhang H, Niu Y, Chang S-F (2018) Grounding referring expressions in images by variational context. In: The IEEE conference on computer vision and pattern recognition
40.
Zurück zum Zitat Zhao S, Yao H, Yang Y, Zhang Y (2014) Affective image retrieval via multi-graph learning. In: Proceedings of the 22nd ACM international conference on multimedia, pp 1025–1028. ACM Zhao S, Yao H, Yang Y, Zhang Y (2014) Affective image retrieval via multi-graph learning. In: Proceedings of the 22nd ACM international conference on multimedia, pp 1025–1028. ACM
Metadaten
Titel
3D Model Retrieval Using Bipartite Graph Matching Based on Attention
verfasst von
Shanlin Sun
Yun Li
Yunfeng Xie
Zhicheng Tan
Xing Yao
Rongyao Zhang
Publikationsdatum
08.01.2020
Verlag
Springer US
Erschienen in
Neural Processing Letters / Ausgabe 2/2020
Print ISSN: 1370-4621
Elektronische ISSN: 1573-773X
DOI
https://doi.org/10.1007/s11063-019-10155-0

Weitere Artikel der Ausgabe 2/2020

Neural Processing Letters 2/2020 Zur Ausgabe

Neuer Inhalt