Skip to main content
Top
Published in: Pattern Analysis and Applications 1/2019

08-01-2019 | Theoretical advances

A nonparametric Bayesian learning model using accelerated variational inference and feature selection

Authors: Wentao Fan, Nizar Bouguila, Xin Liu

Published in: Pattern Analysis and Applications | Issue 1/2019

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Developing effective machine learning methods for multimedia data modeling continues to challenge computer vision scientists. The capability of providing effective learning models can have significant impact on various applications. In this work, we propose a nonparametric Bayesian approach to address simultaneously two fundamental problems, namely clustering and feature selection. The approach is based on infinite generalized Dirichlet (GD) mixture models constructed through the framework of Dirichlet process and learned using an accelerated variational algorithm that we have developed. Furthermore, we extend the proposed approach using another nonparametric Bayesian prior, namely Pitman–Yor process, to construct the infinite generalized Dirichlet mixture model. Our experiments, which were conducted through synthetic data sets, the clustering analysis of real-world data sets and a challenging application, namely automatic human action recognition, indicate that the proposed framework provides good modeling and generalization capabilities.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
1.
2.
go back to reference Bentley JL (1975) Multidimensional binary search trees used for associative searching. Commun ACM 18(9):509–517CrossRefMATH Bentley JL (1975) Multidimensional binary search trees used for associative searching. Commun ACM 18(9):509–517CrossRefMATH
4.
go back to reference Bouguila N (2007) Spatial color image databases summarization. In: Proc. of the IEEE international conference on acoustics, speech and signal processing (ICASSP 2007), vol 1, pp I-953–I-956 Bouguila N (2007) Spatial color image databases summarization. In: Proc. of the IEEE international conference on acoustics, speech and signal processing (ICASSP 2007), vol 1, pp I-953–I-956
5.
go back to reference Bouguila N, Ziou D (2004a) Improving content based image retrieval systems using finite multinomial Dirichlet mixture. In: Proc. of the 14th IEEE signal processing society workshop on machine learning for signal processing, pp 23–32 Bouguila N, Ziou D (2004a) Improving content based image retrieval systems using finite multinomial Dirichlet mixture. In: Proc. of the 14th IEEE signal processing society workshop on machine learning for signal processing, pp 23–32
6.
go back to reference Bouguila N, Ziou D (2004b) A powerful finite mixture model based on the generalized Dirichlet distribution: unsupervised learning and applications. In: Proc. of the 17th international conference on pattern recognition (ICPR 2004), vol 1, pp 280–283 Vol 1 Bouguila N, Ziou D (2004b) A powerful finite mixture model based on the generalized Dirichlet distribution: unsupervised learning and applications. In: Proc. of the 17th international conference on pattern recognition (ICPR 2004), vol 1, pp 280–283 Vol 1
7.
go back to reference Bouguila N, Ziou D (2010) A Dirichlet process mixture of generalized Dirichlet distributions for proportional data modeling. IEEE Trans Neural Netw 21(1):107–122CrossRef Bouguila N, Ziou D (2010) A Dirichlet process mixture of generalized Dirichlet distributions for proportional data modeling. IEEE Trans Neural Netw 21(1):107–122CrossRef
8.
go back to reference Boutemedjet S, Bouguila N, Ziou D (2009) A hybrid feature extraction selection approach for high-dimensional non-Gaussian data clustering. IEEE Trans Pattern Anal Mach Intell 31(8):1429–1443CrossRef Boutemedjet S, Bouguila N, Ziou D (2009) A hybrid feature extraction selection approach for high-dimensional non-Gaussian data clustering. IEEE Trans Pattern Anal Mach Intell 31(8):1429–1443CrossRef
9.
go back to reference Constantinopoulos C, Titsias M, Likas A (2006) Bayesian feature and model selection for Gaussian mixture models. IEEE Trans Pattern Anal Mach Intell 28(6):1013–1018CrossRef Constantinopoulos C, Titsias M, Likas A (2006) Bayesian feature and model selection for Gaussian mixture models. IEEE Trans Pattern Anal Mach Intell 28(6):1013–1018CrossRef
10.
go back to reference Fan W, Bouguila N (2013) Variational learning of a Dirichlet process of generalized Dirichlet distributions for simultaneous clustering and feature selection. Pattern Recognit 46(10):2754–2769CrossRefMATH Fan W, Bouguila N (2013) Variational learning of a Dirichlet process of generalized Dirichlet distributions for simultaneous clustering and feature selection. Pattern Recognit 46(10):2754–2769CrossRefMATH
11.
go back to reference Fan X, Cao L, Xu RYD (2015) Dynamic infinite mixed-membership stochastic blockmodel. IEEE Trans Neural Netw Learn Syst 26(9):2072–2085MathSciNetCrossRef Fan X, Cao L, Xu RYD (2015) Dynamic infinite mixed-membership stochastic blockmodel. IEEE Trans Neural Netw Learn Syst 26(9):2072–2085MathSciNetCrossRef
12.
go back to reference Hofmann T (2001) Unsupervised learning by probabilistic latent semantic analysis. Mach Learn 42(1/2):177–196CrossRefMATH Hofmann T (2001) Unsupervised learning by probabilistic latent semantic analysis. Mach Learn 42(1/2):177–196CrossRefMATH
14.
go back to reference Kuehne H, Jhuang H, Garrote E, Poggio T, Serre T (2011) HMDB: a large video database for human motion recognition. In: Proc. of the international conference on computer vision (ICCV), pp 2556–2563 Kuehne H, Jhuang H, Garrote E, Poggio T, Serre T (2011) HMDB: a large video database for human motion recognition. In: Proc. of the international conference on computer vision (ICCV), pp 2556–2563
15.
go back to reference Kurihara K, Welling M, Vlassis N (2006) Accelerated variational Dirichlet process mixtures. In: Proc. of advances in neural information processing systems (NIPS) Kurihara K, Welling M, Vlassis N (2006) Accelerated variational Dirichlet process mixtures. In: Proc. of advances in neural information processing systems (NIPS)
16.
go back to reference Laptev I (2005) On space–time interest points. Int J Comput Vis 64(2/3):107–123CrossRef Laptev I (2005) On space–time interest points. Int J Comput Vis 64(2/3):107–123CrossRef
17.
go back to reference Laptev I, Marszalek M, Schmid C, Rozenfeld B (2008) Learning realistic human actions from movies. In: Proc. of IEEE conference on computer vision and pattern recognition (CVPR), pp 1–8 Laptev I, Marszalek M, Schmid C, Rozenfeld B (2008) Learning realistic human actions from movies. In: Proc. of IEEE conference on computer vision and pattern recognition (CVPR), pp 1–8
18.
go back to reference Law MHC, Figueiredo MAT, Jain AK (2004) Simultaneous feature selection and clustering using mixture models. IEEE Trans Pattern Anal Mach Intell 26(9):1154–1166CrossRef Law MHC, Figueiredo MAT, Jain AK (2004) Simultaneous feature selection and clustering using mixture models. IEEE Trans Pattern Anal Mach Intell 26(9):1154–1166CrossRef
20.
go back to reference Neal RM (2000) Markov chain sampling methods for Dirichlet process mixture models. J Comput Graph Stat 9(2):249–265MathSciNet Neal RM (2000) Markov chain sampling methods for Dirichlet process mixture models. J Comput Graph Stat 9(2):249–265MathSciNet
21.
go back to reference Nguyen NT, Zheng G, Han Z, Zheng R (2011) Device fingerprinting to enhance wireless security using nonparametric Bayesian method. In: Proc. of the IEEE conference on INFOCOM, pp 1404–1412 Nguyen NT, Zheng G, Han Z, Zheng R (2011) Device fingerprinting to enhance wireless security using nonparametric Bayesian method. In: Proc. of the IEEE conference on INFOCOM, pp 1404–1412
22.
go back to reference Pitman J, Yor M (1997) The two-parameter Poisson–Dirichlet distribution derived from a stable subordinator. Ann Probab 25(2):855–900MathSciNetCrossRefMATH Pitman J, Yor M (1997) The two-parameter Poisson–Dirichlet distribution derived from a stable subordinator. Ann Probab 25(2):855–900MathSciNetCrossRefMATH
23.
24.
go back to reference Shyr A, Darrell T, Jordan M, Urtasun R (2011) Supervised hierarchical Pitman–Yor process for natural scene segmentation. In: Proc. of the 2011 IEEE conference on computer vision and pattern recognition (CVPR), pp 2281–2288 Shyr A, Darrell T, Jordan M, Urtasun R (2011) Supervised hierarchical Pitman–Yor process for natural scene segmentation. In: Proc. of the 2011 IEEE conference on computer vision and pattern recognition (CVPR), pp 2281–2288
25.
go back to reference Song Y, Tang S, Zheng YT, Chua TS, Zhang Y, Lin S (2012) Exploring probabilistic localized video representation for human action recognition. Multimedia Tools and Applications 58(3):663–685CrossRef Song Y, Tang S, Zheng YT, Chua TS, Zhang Y, Lin S (2012) Exploring probabilistic localized video representation for human action recognition. Multimedia Tools and Applications 58(3):663–685CrossRef
26.
go back to reference Sudderth EB, Jordan MI (2008) Shared segmentation of natural scenes using dependent Pitman-Yor processes. In: Proc. of Advances in Neural Information Processing Systems (NIPS), pp 1585–1592 Sudderth EB, Jordan MI (2008) Shared segmentation of natural scenes using dependent Pitman-Yor processes. In: Proc. of Advances in Neural Information Processing Systems (NIPS), pp 1585–1592
27.
go back to reference Teh YW (2006) A hierarchical Bayesian language model based on Pitman-Yor processes. In: Proc. of the 21st International Conference on Computational Linguistics and the 44th Annual Meeting of the Association for Computational Linguistics, ACL-44, pp 985–992 Teh YW (2006) A hierarchical Bayesian language model based on Pitman-Yor processes. In: Proc. of the 21st International Conference on Computational Linguistics and the 44th Annual Meeting of the Association for Computational Linguistics, ACL-44, pp 985–992
28.
go back to reference Walker SG (2007) Sampling the Dirichlet mixture model with slices. Communications in Statistics- Simulation and Computation 36:45–54MathSciNetCrossRefMATH Walker SG (2007) Sampling the Dirichlet mixture model with slices. Communications in Statistics- Simulation and Computation 36:45–54MathSciNetCrossRefMATH
30.
go back to reference Wang T, Hammoud R, Zhu Z (2014) Ground-based activity recognition at distance and behind wall. In: Proc. of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp 231–236 Wang T, Hammoud R, Zhu Z (2014) Ground-based activity recognition at distance and behind wall. In: Proc. of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp 231–236
31.
go back to reference Wei X, Li C (2012) The infinite student’s t-mixture for robust modeling. Signal Processing 92(1):224–234CrossRef Wei X, Li C (2012) The infinite student’s t-mixture for robust modeling. Signal Processing 92(1):224–234CrossRef
32.
go back to reference Wei X, Yang Z (2012) The infinite student’s t-factor mixture analyzer for robust clustering and classification. Pattern Recognition 45(12):4346–4357CrossRefMATH Wei X, Yang Z (2012) The infinite student’s t-factor mixture analyzer for robust clustering and classification. Pattern Recognition 45(12):4346–4357CrossRefMATH
33.
go back to reference Zhang T, Liu S, Xu C, Lu H (2011) Boosted multi-class semi-supervised learning for human action recognition. Pattern Recognition 44(10):2334–2342CrossRefMATH Zhang T, Liu S, Xu C, Lu H (2011) Boosted multi-class semi-supervised learning for human action recognition. Pattern Recognition 44(10):2334–2342CrossRefMATH
Metadata
Title
A nonparametric Bayesian learning model using accelerated variational inference and feature selection
Authors
Wentao Fan
Nizar Bouguila
Xin Liu
Publication date
08-01-2019
Publisher
Springer London
Published in
Pattern Analysis and Applications / Issue 1/2019
Print ISSN: 1433-7541
Electronic ISSN: 1433-755X
DOI
https://doi.org/10.1007/s10044-018-00767-y

Other articles of this Issue 1/2019

Pattern Analysis and Applications 1/2019 Go to the issue

Premium Partner