Skip to main content
Erschienen in: Neural Processing Letters 2/2014

01.04.2014

Non-Gaussian Data Clustering via Expectation Propagation Learning of Finite Dirichlet Mixture Models and Applications

verfasst von: Wentao Fan, Nizar Bouguila

Erschienen in: Neural Processing Letters | Ausgabe 2/2014

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Learning appropriate statistical models is a fundamental data analysis task which has been the topic of continuing interest. Recently, finite Dirichlet mixture models have proved to be an effective and flexible model learning technique in several machine learning and data mining applications. In this article, the problem of learning and selecting finite Dirichlet mixture models is addressed using an expectation propagation (EP) inference framework. Within the proposed EP learning method, for finite mixture models, all the involved parameters and the model complexity (i.e. the number of mixture components), can be evaluated simultaneously in a single optimization framework. Extensive simulations using synthetic data along with two challenging real-world applications involving automatic image annotation and human action videos categorization demonstrate that our approach is able to achieve better results than comparable techniques.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Fußnoten
1
The complete source code of this work is available upon request.
 
3
WordNet is a large lexical database for English, which groups English words into sets of cognitive synonyms called synsets.
 
Literatur
1.
Zurück zum Zitat Blank M, Gorelick L, Shechtman E, Irani M, Basri R (2005) Actions as Space-Time Shapes. In: Proc. of the IEEE International Conference on Computer Vision (ICCV), pp 1395–1402. Blank M, Gorelick L, Shechtman E, Irani M, Basri R (2005) Actions as Space-Time Shapes. In: Proc. of the IEEE International Conference on Computer Vision (ICCV), pp 1395–1402.
2.
Zurück zum Zitat Bouguila N, Ziou D (2005) Using unsupervised learning of a finite dirichlet mixture model to improve pattern recognition applications. Pattern Recognit Lett 26(12):1916–1925CrossRef Bouguila N, Ziou D (2005) Using unsupervised learning of a finite dirichlet mixture model to improve pattern recognition applications. Pattern Recognit Lett 26(12):1916–1925CrossRef
3.
Zurück zum Zitat Bouguila N, Ziou D (2006a) Online clustering via finite mixtures of dirichlet and minimum message length. Eng Appl Artif Intell 19(4):371–379CrossRef Bouguila N, Ziou D (2006a) Online clustering via finite mixtures of dirichlet and minimum message length. Eng Appl Artif Intell 19(4):371–379CrossRef
4.
Zurück zum Zitat Bouguila N, Ziou D (2006b) Unsupervised selection of a finite dirichlet mixture model: an MML-based approach. IEEE Trans Knowl Data Eng 18(8):993–1009CrossRef Bouguila N, Ziou D (2006b) Unsupervised selection of a finite dirichlet mixture model: an MML-based approach. IEEE Trans Knowl Data Eng 18(8):993–1009CrossRef
5.
Zurück zum Zitat Bouguila N, Ziou D, Vaillancourt J (2004) Unsupervised learning of a finite mixture model based on the dirichlet distribution and its application. IEEE Trans Image Process 13(11):1533–1543CrossRef Bouguila N, Ziou D, Vaillancourt J (2004) Unsupervised learning of a finite mixture model based on the dirichlet distribution and its application. IEEE Trans Image Process 13(11):1533–1543CrossRef
6.
Zurück zum Zitat Chang E (2003) CBSA: content-based soft annotation for multinomial image retrieval using bayes point machines. IEEE Trans Circuit Syst Video Technol 13(1):26–38CrossRef Chang E (2003) CBSA: content-based soft annotation for multinomial image retrieval using bayes point machines. IEEE Trans Circuit Syst Video Technol 13(1):26–38CrossRef
7.
Zurück zum Zitat Chang S, Dasgupta N, Carin L (2005) A Bayesian approach to unsupervised feature selection and density estimation using expectation propagation. In: Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp 1043–1050 Chang S, Dasgupta N, Carin L (2005) A Bayesian approach to unsupervised feature selection and density estimation using expectation propagation. In: Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp 1043–1050
8.
Zurück zum Zitat Datta R, Ge W, Li J, Wang JZ (2006) Toward bridging the annotation-retrieval gap in image search by a generative modeling approach. In: Proceedings of the 14th annual ACM international conference on multimedia (MM), ACM, pp 977–986 Datta R, Ge W, Li J, Wang JZ (2006) Toward bridging the annotation-retrieval gap in image search by a generative modeling approach. In: Proceedings of the 14th annual ACM international conference on multimedia (MM), ACM, pp 977–986
9.
Zurück zum Zitat Dollár P, Rabaud V, Cottrell G, Belongie S (2005) Behavior recognition via sparse spatio-temporal feature. In: Proceedings of the IEEE international workshop on visual surveillance and performance evaluation of tracking and surveillance (VS-PETS), pp 65–72 Dollár P, Rabaud V, Cottrell G, Belongie S (2005) Behavior recognition via sparse spatio-temporal feature. In: Proceedings of the IEEE international workshop on visual surveillance and performance evaluation of tracking and surveillance (VS-PETS), pp 65–72
10.
Zurück zum Zitat Fan J, Gao Y, Luo H, Xu G (2005) Statistical modeling and conceptualization of natural images. Pattern Recognit 38:865–885CrossRef Fan J, Gao Y, Luo H, Xu G (2005) Statistical modeling and conceptualization of natural images. Pattern Recognit 38:865–885CrossRef
11.
Zurück zum Zitat Figueiredo M, Jain A (2002) Unsupervised learning of finite mixture models. IEEE Trans Pattern Anal Mach Intell 24(3):381–396CrossRef Figueiredo M, Jain A (2002) Unsupervised learning of finite mixture models. IEEE Trans Pattern Anal Mach Intell 24(3):381–396CrossRef
12.
Zurück zum Zitat Heskes T, Zoeter O (2002) Expectation propagation for approximate inference in dynamic Bayesian networks. In: Proceedings of the conference on uncertainty in artificial intelligence (UAI), pp 216–223 Heskes T, Zoeter O (2002) Expectation propagation for approximate inference in dynamic Bayesian networks. In: Proceedings of the conference on uncertainty in artificial intelligence (UAI), pp 216–223
13.
Zurück zum Zitat Hofmann T (2001) Unsupervised learning by probabilistic latent semantic analysis. Mach Learn 42(1/2):177–196CrossRefMATH Hofmann T (2001) Unsupervised learning by probabilistic latent semantic analysis. Mach Learn 42(1/2):177–196CrossRefMATH
14.
Zurück zum Zitat Ke Y, Sukthankar R (2004) PCA-SIFT: a more distinctive representation for local image descriptors. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 506–513 Ke Y, Sukthankar R (2004) PCA-SIFT: a more distinctive representation for local image descriptors. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 506–513
15.
Zurück zum Zitat Laptev I (2005) On space-time interest points. Int J Comput Vis 64(2/3):107–123CrossRef Laptev I (2005) On space-time interest points. Int J Comput Vis 64(2/3):107–123CrossRef
16.
Zurück zum Zitat Laptev I, Marszalek M, Schmid C, Rozenfeld B (2008) Learning realistic human actions from movies. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 1–8 Laptev I, Marszalek M, Schmid C, Rozenfeld B (2008) Learning realistic human actions from movies. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 1–8
17.
Zurück zum Zitat Law MHC, Figueiredo MAT, Jain AK (2004) Simultaneous feature selection and clustering using mixture models. IEEE Trans Pattern Anal Mach Intell 26(9):1154–1166CrossRef Law MHC, Figueiredo MAT, Jain AK (2004) Simultaneous feature selection and clustering using mixture models. IEEE Trans Pattern Anal Mach Intell 26(9):1154–1166CrossRef
18.
Zurück zum Zitat Leacock C, Chodorow M (1998) In: Fellbaum C (Ed) WordNet: an electronic lexical database. MIT Press, pp 305–332 Leacock C, Chodorow M (1998) In: Fellbaum C (Ed) WordNet: an electronic lexical database. MIT Press, pp 305–332
19.
Zurück zum Zitat Liu J, Luo J, Shah M (2009) Recognizing realistic actions from videos “In The Wild”. In: Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp 1996–2003 Liu J, Luo J, Shah M (2009) Recognizing realistic actions from videos “In The Wild”. In: Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp 1996–2003
20.
Zurück zum Zitat Luo J, Savakis AE, Singhal A (2005) A Bayesian network-based framework for semantic image understanding. Pattern Recognit 38:919–934CrossRef Luo J, Savakis AE, Singhal A (2005) A Bayesian network-based framework for semantic image understanding. Pattern Recognit 38:919–934CrossRef
21.
Zurück zum Zitat Ma Z, Leijon A (2010) Expectation propagation for estimating the parameters of the beta distribution. In: Proceedings of IEEE international conference on acoustics speech and signal processing (ICASSP), pp 2082–2085 Ma Z, Leijon A (2010) Expectation propagation for estimating the parameters of the beta distribution. In: Proceedings of IEEE international conference on acoustics speech and signal processing (ICASSP), pp 2082–2085
22.
Zurück zum Zitat Maybeck PS (1982) Stochastic models, estimation and control. Academic Press Maybeck PS (1982) Stochastic models, estimation and control. Academic Press
24.
Zurück zum Zitat Mikolajczyk K, Schmid C (2004) Scale and affine invariant interest point detectors. Int J Comput Vis 60:63–86CrossRef Mikolajczyk K, Schmid C (2004) Scale and affine invariant interest point detectors. Int J Comput Vis 60:63–86CrossRef
25.
Zurück zum Zitat Miller GA (1995) WordNet: a lexical database for English. Commun ACM 38:39–41CrossRef Miller GA (1995) WordNet: a lexical database for English. Commun ACM 38:39–41CrossRef
26.
Zurück zum Zitat Minka T (2001) Expectation propagation for approximate Bayesian inference. In: Proceedings of the conference on uncertainty in artificial intelligence (UAI), pp 362–369 Minka T (2001) Expectation propagation for approximate Bayesian inference. In: Proceedings of the conference on uncertainty in artificial intelligence (UAI), pp 362–369
27.
Zurück zum Zitat Minka T, Lafferty J (2002) Expectation-propagation for the generative aspect model. In: Proceedings of the conference on uncertainty in artificial intelligence (UAI), pp 352–359 Minka T, Lafferty J (2002) Expectation-propagation for the generative aspect model. In: Proceedings of the conference on uncertainty in artificial intelligence (UAI), pp 352–359
28.
Zurück zum Zitat Naphade MR, Huang TS (2001) A probabilistic framework for semantic video indexing, filtering, and retrieval. IEEE Trans Multimed 3:141–151CrossRef Naphade MR, Huang TS (2001) A probabilistic framework for semantic video indexing, filtering, and retrieval. IEEE Trans Multimed 3:141–151CrossRef
29.
Zurück zum Zitat Rodriguez M, Ahmed J, Shah M (2008) Action mach a spatio-temporal maximum average correlation height filter for action recognition. In: Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp 1–8 Rodriguez M, Ahmed J, Shah M (2008) Action mach a spatio-temporal maximum average correlation height filter for action recognition. In: Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp 1–8
30.
Zurück zum Zitat Russell B, Torralba A, Murphy K, Freeman W (2008) LabelMe: a database and Web-based tool for image annotation. Int J Comput Vis 77:157–173CrossRef Russell B, Torralba A, Murphy K, Freeman W (2008) LabelMe: a database and Web-based tool for image annotation. Int J Comput Vis 77:157–173CrossRef
31.
Zurück zum Zitat Schüldt C, Laptev I, Caputo B (2004) Recognizing human actions: a local SVM approach. In: Proceedings of the 17th international conference on pattern recognition (ICPR), pp 32–36 Schüldt C, Laptev I, Caputo B (2004) Recognizing human actions: a local SVM approach. In: Proceedings of the 17th international conference on pattern recognition (ICPR), pp 32–36
32.
Zurück zum Zitat Zhao R, Grosky WI (2000) From features to semantics: some preliminary results. In: Proceedings of the IEEE international conference on multimedia and expo (ICME). IEEE Computer Society, pp 679–682 Zhao R, Grosky WI (2000) From features to semantics: some preliminary results. In: Proceedings of the IEEE international conference on multimedia and expo (ICME). IEEE Computer Society, pp 679–682
33.
Zurück zum Zitat Zhong D, Zhang H, Chang SF (1997) Clustering methods for video browsing and annotation. In: Proceedings of the SPIE conference on storage and retrieval for video and image databases, pp 239–246 Zhong D, Zhang H, Chang SF (1997) Clustering methods for video browsing and annotation. In: Proceedings of the SPIE conference on storage and retrieval for video and image databases, pp 239–246
Metadaten
Titel
Non-Gaussian Data Clustering via Expectation Propagation Learning of Finite Dirichlet Mixture Models and Applications
verfasst von
Wentao Fan
Nizar Bouguila
Publikationsdatum
01.04.2014
Verlag
Springer US
Erschienen in
Neural Processing Letters / Ausgabe 2/2014
Print ISSN: 1370-4621
Elektronische ISSN: 1573-773X
DOI
https://doi.org/10.1007/s11063-013-9293-x

Weitere Artikel der Ausgabe 2/2014

Neural Processing Letters 2/2014 Zur Ausgabe

Neuer Inhalt