Skip to main content
Erschienen in: Soft Computing 24/2020

26.06.2020 | Methodologies and Application

A particle swarm optimization-based feature selection for unsupervised transfer learning

verfasst von: Rakesh Kumar Sanodiya, Mrinalini Tiwari, Jimson Mathew, Sriparna Saha, Subhajyoti Saha

Erschienen in: Soft Computing | Ausgabe 24/2020

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Transfer learning (TL) method has captured an attractive presence because it facilitates the learning ability in the target domain by acquiring knowledge from well-established source domains. To gain strong knowledge from the source domain, it is important to narrow down the distribution difference between the source and the target domains. For this purpose, it is necessary to consider the objectives such as preserving the discriminative information, preserving the original similarity of the source and the target domain data, maximizing the variance of the target domain, and preserving marginal and conditional distribution at the same time. Furthermore, some existing TL methods use only original feature data, so there is a threat of degenerated feature transformation. To overcome all these limitations, in this paper, a novel feature selection-based transfer learning approach using particle swarm optimization (PSO) for unsupervised transfer learning (FSUTL-PSO) is implemented. In FSUTL-PSO, we incorporate all such objectives into one fitness function and select common good features from the source and target domains based on the fitness function for eliminating the threat of degenerated features. Extensive experiments have been done on all possible tasks of Office+Caltech and PIE Face datasets and our proposed method FSUTL-PSO has shown significant improvement over the existing transfer or non-transfer learning methods.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Literatur
Zurück zum Zitat Amid E, Gionis A, Ukkonen A (2015) A kernel-learning approach to semi-supervised clustering with relative distance comparisons. In: Joint European conference on machine learning and knowledge discovery in databases. Springer, New York, pp 219–234 Amid E, Gionis A, Ukkonen A (2015) A kernel-learning approach to semi-supervised clustering with relative distance comparisons. In: Joint European conference on machine learning and knowledge discovery in databases. Springer, New York, pp 219–234
Zurück zum Zitat Baktashmotlagh M, Harandi M, Salzmann M (2016) Distribution-matching embedding for visual domain adaptation. J Mach Learn Res 17(1):3760–3789MathSciNetMATH Baktashmotlagh M, Harandi M, Salzmann M (2016) Distribution-matching embedding for visual domain adaptation. J Mach Learn Res 17(1):3760–3789MathSciNetMATH
Zurück zum Zitat Chen J, Liu Y (2011) Locally linear embedding: a survey. Artif Intell Rev 36(1):29–48CrossRef Chen J, Liu Y (2011) Locally linear embedding: a survey. Artif Intell Rev 36(1):29–48CrossRef
Zurück zum Zitat Eberhart R, Kennedy J (1995) A new optimizer using particle swarm theory. In: MHS’95 Proceedings of the sixth international symposium on micro machine and human science. IEEE, pp 39–43 Eberhart R, Kennedy J (1995) A new optimizer using particle swarm theory. In: MHS’95 Proceedings of the sixth international symposium on micro machine and human science. IEEE, pp 39–43
Zurück zum Zitat Fernando B, Habrard A, Sebban M, Tuytelaars T (2013) Unsupervised visual domain adaptation using subspace alignment. In: Proceedings of the IEEE international conference on computer vision, pp 2960–2967 Fernando B, Habrard A, Sebban M, Tuytelaars T (2013) Unsupervised visual domain adaptation using subspace alignment. In: Proceedings of the IEEE international conference on computer vision, pp 2960–2967
Zurück zum Zitat Gong B, Shi Y, Sha F, Grauman K (2012) Geodesic flow kernel for unsupervised domain adaptation. In 2012 IEEE conference on computer vision and pattern recognition. IEEE, pp 2066–2073 Gong B, Shi Y, Sha F, Grauman K (2012) Geodesic flow kernel for unsupervised domain adaptation. In 2012 IEEE conference on computer vision and pattern recognition. IEEE, pp 2066–2073
Zurück zum Zitat Gretton A, Borgwardt KM, Rasch MJ, Schölkopf B, Smola A (2012) A kernel two-sample test. J Mach Learn Res 13(Mar):723–773MathSciNetMATH Gretton A, Borgwardt KM, Rasch MJ, Schölkopf B, Smola A (2012) A kernel two-sample test. J Mach Learn Res 13(Mar):723–773MathSciNetMATH
Zurück zum Zitat He X, Niyogi P (2004) Locality preserving projections. In: Advances in neural information processing systems, pp 153–160 He X, Niyogi P (2004) Locality preserving projections. In: Advances in neural information processing systems, pp 153–160
Zurück zum Zitat Herath S, Harandi M, Porikli F (2017) Learning an invariant hilbert space for domain adaptation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3845–3854 Herath S, Harandi M, Porikli F (2017) Learning an invariant hilbert space for domain adaptation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3845–3854
Zurück zum Zitat Hofmann T, Schölkopf B, Smola AJ (2008) Kernel methods in machine learning. Ann Stat 1171–1220 Hofmann T, Schölkopf B, Smola AJ (2008) Kernel methods in machine learning. Ann Stat 1171–1220
Zurück zum Zitat Huang K-K, Dai D-Q, Ren C-X (2017) Regularized coplanar discriminant analysis for dimensionality reduction. Pattern Recognit 62:87–98MATHCrossRef Huang K-K, Dai D-Q, Ren C-X (2017) Regularized coplanar discriminant analysis for dimensionality reduction. Pattern Recognit 62:87–98MATHCrossRef
Zurück zum Zitat Hull JJ (1994) A database for handwritten text recognition research. IEEE Trans Pattern Anal Mach Intell 16(5):550–554CrossRef Hull JJ (1994) A database for handwritten text recognition research. IEEE Trans Pattern Anal Mach Intell 16(5):550–554CrossRef
Zurück zum Zitat Ionescu B, Lupu M, Rohm M, Gînsca AL, Müller H (2018) Datasets column: diversity and credibility for social images and image retrieval. ACM SIGMultimedia Records 9(3):7CrossRef Ionescu B, Lupu M, Rohm M, Gînsca AL, Müller H (2018) Datasets column: diversity and credibility for social images and image retrieval. ACM SIGMultimedia Records 9(3):7CrossRef
Zurück zum Zitat Kennedy J (1997) The particle swarm: social adaptation of knowledge. In: Proceedings of 1997 IEEE international conference on evolutionary computation (ICEC’97). IEEE, pp 303–308 Kennedy J (1997) The particle swarm: social adaptation of knowledge. In: Proceedings of 1997 IEEE international conference on evolutionary computation (ICEC’97). IEEE, pp 303–308
Zurück zum Zitat Kennedy J, Eberhart R (1995) Particle swarm optimization (pso). In: Proceedings of IEEE international conference on neural networks, Perth, Australia, pp 1942–1948 Kennedy J, Eberhart R (1995) Particle swarm optimization (pso). In: Proceedings of IEEE international conference on neural networks, Perth, Australia, pp 1942–1948
Zurück zum Zitat Li H, Jiang T, Zhang K (2004) Efficient and robust feature extraction by maximum margin criterion. In: Advances in neural information processing systems, pp 97–104 Li H, Jiang T, Zhang K (2004) Efficient and robust feature extraction by maximum margin criterion. In: Advances in neural information processing systems, pp 97–104
Zurück zum Zitat Long M, Wang J, Ding G, Pan SJ, Philip SY (2014a) Adaptation regularization: a general framework for transfer learning. IEEE Trans Knowl Data Eng 26(5):1076–1089CrossRef Long M, Wang J, Ding G, Pan SJ, Philip SY (2014a) Adaptation regularization: a general framework for transfer learning. IEEE Trans Knowl Data Eng 26(5):1076–1089CrossRef
Zurück zum Zitat Long M, Wang J, Ding G, Sun J, Yu PS (2013) Transfer feature learning with joint distribution adaptation. In: Proceedings of the IEEE international conference on computer vision, pp 2200–2207 Long M, Wang J, Ding G, Sun J, Yu PS (2013) Transfer feature learning with joint distribution adaptation. In: Proceedings of the IEEE international conference on computer vision, pp 2200–2207
Zurück zum Zitat Long M, Wang J, Ding G, Sun J, Yu PS (2014b) Transfer joint matching for unsupervised domain adaptation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1410–1417 Long M, Wang J, Ding G, Sun J, Yu PS (2014b) Transfer joint matching for unsupervised domain adaptation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1410–1417
Zurück zum Zitat Luo L, Chen L, Hu S, Lu Y, Wang X (2017) Discriminative and geometry aware unsupervised domain adaptation. arXiv preprint arXiv:1712.10042 Luo L, Chen L, Hu S, Lu Y, Wang X (2017) Discriminative and geometry aware unsupervised domain adaptation. arXiv preprint arXiv:​1712.​10042
Zurück zum Zitat Nguyen BH, Xue B, Andreae P (2018) A particle swarm optimization based feature selection approach to transfer learning in classification. In: Proceedings of the genetic and evolutionary computation conference, pp 37–44 Nguyen BH, Xue B, Andreae P (2018) A particle swarm optimization based feature selection approach to transfer learning in classification. In: Proceedings of the genetic and evolutionary computation conference, pp 37–44
Zurück zum Zitat Nie F, Xiang S, Jia Y, Zhang C (2009) Semi-supervised orthogonal discriminant analysis via label propagation. Pattern Recognit 42(11):2615–2627MATHCrossRef Nie F, Xiang S, Jia Y, Zhang C (2009) Semi-supervised orthogonal discriminant analysis via label propagation. Pattern Recognit 42(11):2615–2627MATHCrossRef
Zurück zum Zitat Pan SJ, Tsang IW, Kwok JT, Yang Q (2011) Domain adaptation via transfer component analysis. IEEE Trans Neural Netw 22(2):199–210CrossRef Pan SJ, Tsang IW, Kwok JT, Yang Q (2011) Domain adaptation via transfer component analysis. IEEE Trans Neural Netw 22(2):199–210CrossRef
Zurück zum Zitat Patel VM, Gopalan R, Li R, Chellappa R (2015) Visual domain adaptation: A survey of recent advances. IEEE Signal Processing Mag 32(3):53–69CrossRef Patel VM, Gopalan R, Li R, Chellappa R (2015) Visual domain adaptation: A survey of recent advances. IEEE Signal Processing Mag 32(3):53–69CrossRef
Zurück zum Zitat Sanodiya RK, Mathew J, Saha S, Thalakottur MD (2019) A new transfer learning algorithm in semi-supervised setting. IEEE Access 7:42956–42967CrossRef Sanodiya RK, Mathew J, Saha S, Thalakottur MD (2019) A new transfer learning algorithm in semi-supervised setting. IEEE Access 7:42956–42967CrossRef
Zurück zum Zitat Satpal, S. and Sarawagi, S. (2007). Domain adaptation of conditional probability models via feature subsetting. In: European conference on principles of data mining and knowledge discovery. Springer, New York, pp 224–235 Satpal, S. and Sarawagi, S. (2007). Domain adaptation of conditional probability models via feature subsetting. In: European conference on principles of data mining and knowledge discovery. Springer, New York, pp 224–235
Zurück zum Zitat Shao M, Kit D, Fu Y (2014) Generalized transfer subspace learning through low-rank constraint. Int J Comput Vis 109(1–2):74–93MathSciNetMATHCrossRef Shao M, Kit D, Fu Y (2014) Generalized transfer subspace learning through low-rank constraint. Int J Comput Vis 109(1–2):74–93MathSciNetMATHCrossRef
Zurück zum Zitat Shi L, Zhang L, Zhao L, Zhang L, Li P, Wu D (2016) Adaptive Laplacian eigenmap-based dimension reduction for ocean target discrimination. IEEE Geosci Remote Sens Lett 13(7):902–906CrossRef Shi L, Zhang L, Zhao L, Zhang L, Li P, Wu D (2016) Adaptive Laplacian eigenmap-based dimension reduction for ocean target discrimination. IEEE Geosci Remote Sens Lett 13(7):902–906CrossRef
Zurück zum Zitat Shu L, Latecki LJ (2015) Transductive domain adaptation with affinity learning. In: Proceedings of the 24th ACM international on conference on information and knowledge management. ACM, pp 1903–1906 Shu L, Latecki LJ (2015) Transductive domain adaptation with affinity learning. In: Proceedings of the 24th ACM international on conference on information and knowledge management. ACM, pp 1903–1906
Zurück zum Zitat Si S, Tao D, Geng B (2010) Bregman divergence-based regularization for transfer subspace learning. IEEE Trans Knowl Data Eng 22(7):929–942CrossRef Si S, Tao D, Geng B (2010) Bregman divergence-based regularization for transfer subspace learning. IEEE Trans Knowl Data Eng 22(7):929–942CrossRef
Zurück zum Zitat Sun B, Feng J, Saenko K (2016) Return of frustratingly easy domain adaptation. In: Thirtieth AAAI conference on artificial intelligence Sun B, Feng J, Saenko K (2016) Return of frustratingly easy domain adaptation. In: Thirtieth AAAI conference on artificial intelligence
Zurück zum Zitat Sun B, Saenko K (2015) Subspace distribution alignment for unsupervised domain adaptation. BMVC 4:24–1 Sun B, Saenko K (2015) Subspace distribution alignment for unsupervised domain adaptation. BMVC 4:24–1
Zurück zum Zitat Uzair M, Mian A (2017) Blind domain adaptation with augmented extreme learning machine features. IEEE Trans Cybern 47(3):651–660CrossRef Uzair M, Mian A (2017) Blind domain adaptation with augmented extreme learning machine features. IEEE Trans Cybern 47(3):651–660CrossRef
Zurück zum Zitat Wang H, Wang W, Zhang C, Xu F (2014) Cross-domain metric learning based on information theory. In: Twenty-Eighth AAAI conference on artificial intelligence Wang H, Wang W, Zhang C, Xu F (2014) Cross-domain metric learning based on information theory. In: Twenty-Eighth AAAI conference on artificial intelligence
Zurück zum Zitat Wang J, Chen Y, Hao S, Feng W, Shen Z (2017) Balanced distribution adaptation for transfer learning. In: 2017 IEEE international conference on data mining (ICDM), pp 1129–1134 Wang J, Chen Y, Hao S, Feng W, Shen Z (2017) Balanced distribution adaptation for transfer learning. In: 2017 IEEE international conference on data mining (ICDM), pp 1129–1134
Zurück zum Zitat Wang J, Feng W, Chen Y, Yu H, Huang M, Yu PS (2018) Visual domain adaptation with manifold embedded distribution alignment. In: 2018 ACM multimedia conference on multimedia conference. ACM, pp 402–410 Wang J, Feng W, Chen Y, Yu H, Huang M, Yu PS (2018) Visual domain adaptation with manifold embedded distribution alignment. In: 2018 ACM multimedia conference on multimedia conference. ACM, pp 402–410
Zurück zum Zitat Wang S, Lu J, Gu X, Du H, Yang J (2016) Semi-supervised linear discriminant analysis for dimension reduction and classification. Pattern Recognit 57:179–189MATHCrossRef Wang S, Lu J, Gu X, Du H, Yang J (2016) Semi-supervised linear discriminant analysis for dimension reduction and classification. Pattern Recognit 57:179–189MATHCrossRef
Zurück zum Zitat Weinberger KQ, Saul LK (2009) Distance metric learning for large margin nearest neighbor classification. J Mach Learn Res 10(Feb):207–244MATH Weinberger KQ, Saul LK (2009) Distance metric learning for large margin nearest neighbor classification. J Mach Learn Res 10(Feb):207–244MATH
Zurück zum Zitat Xu Y, Fang X, Wu J, Li X, Zhang D (2016) Discriminative transfer subspace learning via low-rank and sparse representation. IEEE Trans Image Process 25(2):850–863MathSciNetMATHCrossRef Xu Y, Fang X, Wu J, Li X, Zhang D (2016) Discriminative transfer subspace learning via low-rank and sparse representation. IEEE Trans Image Process 25(2):850–863MathSciNetMATHCrossRef
Zurück zum Zitat Xu Y, Pan SJ, Xiong H, Wu Q, Luo R, Min H, Song H (2017) A unified framework for metric transfer learning. IEEE Trans Knowl Data Eng 29(6):1158–1171CrossRef Xu Y, Pan SJ, Xiong H, Wu Q, Luo R, Min H, Song H (2017) A unified framework for metric transfer learning. IEEE Trans Knowl Data Eng 29(6):1158–1171CrossRef
Zurück zum Zitat Yan K, Kou L, Zhang D (2018) Learning domain-invariant subspace using domain features and independence maximization. IEEE Trans Cybern 48(1):288–299CrossRef Yan K, Kou L, Zhang D (2018) Learning domain-invariant subspace using domain features and independence maximization. IEEE Trans Cybern 48(1):288–299CrossRef
Zurück zum Zitat Zhang J, Li W, Ogunbona P (2017a) Joint geometrical and statistical alignment for visual domain adaptation. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 1859–1867 Zhang J, Li W, Ogunbona P (2017a) Joint geometrical and statistical alignment for visual domain adaptation. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 1859–1867
Metadaten
Titel
A particle swarm optimization-based feature selection for unsupervised transfer learning
verfasst von
Rakesh Kumar Sanodiya
Mrinalini Tiwari
Jimson Mathew
Sriparna Saha
Subhajyoti Saha
Publikationsdatum
26.06.2020
Verlag
Springer Berlin Heidelberg
Erschienen in
Soft Computing / Ausgabe 24/2020
Print ISSN: 1432-7643
Elektronische ISSN: 1433-7479
DOI
https://doi.org/10.1007/s00500-020-05105-1

Weitere Artikel der Ausgabe 24/2020

Soft Computing 24/2020 Zur Ausgabe