Skip to main content
Erschienen in: International Journal of Machine Learning and Cybernetics 6/2013

01.12.2013 | Original Article

A variational Bayesian framework for group feature selection

verfasst von: Niranjan Subrahmanya, Yung C. Shin

Erschienen in: International Journal of Machine Learning and Cybernetics | Ausgabe 6/2013

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

In many machine learning and pattern analysis applications, grouping of features during model development and the selection of a small number of relevant groups can be useful to improve the interpretability of the learned parameters. Although this problem has been receiving a significant amount of attention lately, most of the approaches require the manual tuning of one or more hyper-parameters. In order to overcome this drawback, this work presents a novel hierarchical Bayesian formulation of a generalized linear model and estimates the posterior distribution of the parameters and hyper-parameters of the model within a completely Bayesian paradigm based on variational inference. All the required computations are analytically tractable. The performance and applicability of the proposed framework is demonstrated on synthetic and real world examples.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Weitere Produktempfehlungen anzeigen
Literatur
1.
Zurück zum Zitat Hall DL (1992) Mathematical techniques in multisensor data fusion. Artech House Inc., Boston Hall DL (1992) Mathematical techniques in multisensor data fusion. Artech House Inc., Boston
2.
Zurück zum Zitat Subrahmanya N, Shin YC (2010) Sparse multiple kernel learning for signal processing applications. IEEE Trans Pattern Anal Mach Intell 32(5):788–798CrossRef Subrahmanya N, Shin YC (2010) Sparse multiple kernel learning for signal processing applications. IEEE Trans Pattern Anal Mach Intell 32(5):788–798CrossRef
3.
Zurück zum Zitat Guo G, Chen S, Chen L (2012) Soft subspace clustering with an improved feature weight self-adjustment mechanism. Int J Mach Learn Cybern 3(1):39–49CrossRef Guo G, Chen S, Chen L (2012) Soft subspace clustering with an improved feature weight self-adjustment mechanism. Int J Mach Learn Cybern 3(1):39–49CrossRef
4.
Zurück zum Zitat Qin Y, Zheng D, Zhao T (2012) Research on search results optimization technology with category features integration. Int J Mach Learn Cybern 3(1):71–76CrossRef Qin Y, Zheng D, Zhao T (2012) Research on search results optimization technology with category features integration. Int J Mach Learn Cybern 3(1):71–76CrossRef
5.
Zurück zum Zitat Pan D, Jian G, Wurtele ES, Dickerson JA (2005) Modeling gene expression networks using fuzzy logic. IEEE Trans Syst Man Cybern Part B 35(6):1351–1359CrossRef Pan D, Jian G, Wurtele ES, Dickerson JA (2005) Modeling gene expression networks using fuzzy logic. IEEE Trans Syst Man Cybern Part B 35(6):1351–1359CrossRef
6.
7.
Zurück zum Zitat Guyon I, Elisseeff A (2003) An introduction to variable and feature selection. J Mach Leran Res 3:1157–1182MATH Guyon I, Elisseeff A (2003) An introduction to variable and feature selection. J Mach Leran Res 3:1157–1182MATH
9.
Zurück zum Zitat Tibshirani R (1996) Regression shrinkage and selection via the lasso. J Royal Stat Soc Ser B 58(1):267–288MathSciNetMATH Tibshirani R (1996) Regression shrinkage and selection via the lasso. J Royal Stat Soc Ser B 58(1):267–288MathSciNetMATH
10.
Zurück zum Zitat Weston J, Elisseeff A, Scholkopf B, Tipping M (2003) Use of the zero-norm with linear models and kernel methods. J Mach Leran Res 3:1439–1461MathSciNetMATH Weston J, Elisseeff A, Scholkopf B, Tipping M (2003) Use of the zero-norm with linear models and kernel methods. J Mach Leran Res 3:1439–1461MathSciNetMATH
11.
Zurück zum Zitat Figueiredo MAT (2003) Adaptive sparseness for supervised learning. IEEE Trans Pattern Anal Mach Intell 25(9):1150–1159MathSciNetCrossRef Figueiredo MAT (2003) Adaptive sparseness for supervised learning. IEEE Trans Pattern Anal Mach Intell 25(9):1150–1159MathSciNetCrossRef
12.
Zurück zum Zitat Lal TN, Schroder M, Hinterberger T, Weston J, Bogdan M, Birbaumer N, Scholkopf B (2004) Support vector channel selection in BCI. IEEE Trans Biomed Eng 51(6):1003–1010CrossRef Lal TN, Schroder M, Hinterberger T, Weston J, Bogdan M, Birbaumer N, Scholkopf B (2004) Support vector channel selection in BCI. IEEE Trans Biomed Eng 51(6):1003–1010CrossRef
13.
14.
Zurück zum Zitat Similä T, Tikka J (2007) Input selection and shrinkage in multiresponse linear regression. Comput Stat Data Anal 52:406–422CrossRefMATH Similä T, Tikka J (2007) Input selection and shrinkage in multiresponse linear regression. Comput Stat Data Anal 52:406–422CrossRefMATH
15.
Zurück zum Zitat Wang L, Chen G, Li H (2007) Group SCAD regression analysis for microarray time course gene expression data. Bioinformatics 23(12):1486–1494MathSciNetCrossRef Wang L, Chen G, Li H (2007) Group SCAD regression analysis for microarray time course gene expression data. Bioinformatics 23(12):1486–1494MathSciNetCrossRef
16.
17.
Zurück zum Zitat Meier L, van de Geer S, Buhlmann P (2006) The group lasso for logistic regression. In: Technical Report, Eidgenössische Technische Hochschule Meier L, van de Geer S, Buhlmann P (2006) The group lasso for logistic regression. In: Technical Report, Eidgenössische Technische Hochschule
18.
Zurück zum Zitat Stoeckel J, Fung G (2005) SVM feature selection for classification of SPECT images of Alzheimer’s disease using spatial information. In: IEEE Conference on Data Mining Stoeckel J, Fung G (2005) SVM feature selection for classification of SPECT images of Alzheimer’s disease using spatial information. In: IEEE Conference on Data Mining
19.
Zurück zum Zitat Bach F, Jenatton R, Mairal J, Obozinski G (2011) Convex optimization with sparsity-inducing norms. In: Sra S, Nowozin S, Wright SJ (eds) Optimization for Machine Learning. MIT Press, Cambridge Bach F, Jenatton R, Mairal J, Obozinski G (2011) Convex optimization with sparsity-inducing norms. In: Sra S, Nowozin S, Wright SJ (eds) Optimization for Machine Learning. MIT Press, Cambridge
20.
Zurück zum Zitat Jenatton R, Mairal J, Obozinski G, Bach F (2010) Proximal methods for sparse hierarchical dictionary learning. In: International Conference on Machine Learning Jenatton R, Mairal J, Obozinski G, Bach F (2010) Proximal methods for sparse hierarchical dictionary learning. In: International Conference on Machine Learning
21.
Zurück zum Zitat Bishop CM, Tipping ME (2000) Variational relevance vector machines. In: Boutilier C, Goldzmidt M (eds) Uncertainty in artificial intelligence. Morgan Kaufmann, California, pp 46–53 Bishop CM, Tipping ME (2000) Variational relevance vector machines. In: Boutilier C, Goldzmidt M (eds) Uncertainty in artificial intelligence. Morgan Kaufmann, California, pp 46–53
22.
Zurück zum Zitat Tipping ME (2001) Sparse bayesian learning and the relevance vector machine. J Mach Leran Res 1(3):211–244MathSciNetMATH Tipping ME (2001) Sparse bayesian learning and the relevance vector machine. J Mach Leran Res 1(3):211–244MathSciNetMATH
23.
Zurück zum Zitat Kullback S (1959) Information theory and statistics. Wiley, New YorkMATH Kullback S (1959) Information theory and statistics. Wiley, New YorkMATH
25.
Zurück zum Zitat Martens H, Martens M (2001) Multivariate analysis of quality: an introduction. Wiley, New York Martens H, Martens M (2001) Multivariate analysis of quality: an introduction. Wiley, New York
26.
Zurück zum Zitat Subrahmanya N, Shin YC (2008) Automated sensor selection and fusion for monitoring and diagnostics of plunge grinding. J Manufact Sci Eng Trans ASME 130(3):031014CrossRef Subrahmanya N, Shin YC (2008) Automated sensor selection and fusion for monitoring and diagnostics of plunge grinding. J Manufact Sci Eng Trans ASME 130(3):031014CrossRef
27.
Zurück zum Zitat Liu Q, Dinu I, Adewale AJ, Potter JD, Yasui Y (2007) Comparative evaluation of gene-set analysis methods. BMC Bioinformatics 8:431CrossRef Liu Q, Dinu I, Adewale AJ, Potter JD, Yasui Y (2007) Comparative evaluation of gene-set analysis methods. BMC Bioinformatics 8:431CrossRef
Metadaten
Titel
A variational Bayesian framework for group feature selection
verfasst von
Niranjan Subrahmanya
Yung C. Shin
Publikationsdatum
01.12.2013
Verlag
Springer Berlin Heidelberg
Erschienen in
International Journal of Machine Learning and Cybernetics / Ausgabe 6/2013
Print ISSN: 1868-8071
Elektronische ISSN: 1868-808X
DOI
https://doi.org/10.1007/s13042-012-0121-9

Weitere Artikel der Ausgabe 6/2013

International Journal of Machine Learning and Cybernetics 6/2013 Zur Ausgabe

Neuer Inhalt