Skip to main content
Erschienen in: Cognitive Computation 4/2018

21.04.2018

A Joint Unsupervised Cross-Domain Model via Scalable Discriminative Extreme Learning Machine

verfasst von: Boyang Zhang, Yingyi Liu, Haiwen Yuan, Lingjie Sun, Zhao Ma

Erschienen in: Cognitive Computation | Ausgabe 4/2018

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Extreme learning machine (ELM) is a well-known cognitive model, that has been extended to cross-domain tasks. Nonetheless, most existing paradigms that are based on ELM either concern about a case in which a specific number of instances are labelled in the target domain or a learner is trained without sufficient capacity to eliminate the gap between domains. To cope with the scenario in which there are no target labels and to acquire a better adaptive learner, we propose a joint unsupervised cross-domain model via scalable discriminative ELM, which is abbreviated as JUC-SDELM. Within the framework, the scalable factor is integrated into discriminative ELM (DELM) to adjust the output margin, which strengthens the discriminative capacity of the ELM classifier. In addition, we follow the basic strategy of joint distribution adaptation (JDA) to align the subspaces generated by JUC-SDELM in terms of their statistics. The discrepancy across domains is alleviated after a few iterations. Moreover, a metric on the outputs of ELM is utilized to filter unreliable pseudo labels in the target domain, with the aim of eliminating the negative transfer effect. Results are obtained by comparing JUC-SDELM with state-of-the-art baseline methods on 16 cross-domain benchmarks that were constructed based on three combined datasets. Likewise, the outcomes in terms of key parameters are also examined. According to the experiments, our proposed model achieves competitive overall performance.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Pan SJ, Yang Q. A survey on transfer learning. IEEE Trans Knowl Data Eng. 2010;22(10):1345–1359.CrossRef Pan SJ, Yang Q. A survey on transfer learning. IEEE Trans Knowl Data Eng. 2010;22(10):1345–1359.CrossRef
2.
Zurück zum Zitat Gao S, Li H. A cross-domain adaptation method for sentiment classification using probabilistic latent analysis. Proceedings of the 20th ACM international conference on Information and knowledge management; 2011. p. 1047–1052. Gao S, Li H. A cross-domain adaptation method for sentiment classification using probabilistic latent analysis. Proceedings of the 20th ACM international conference on Information and knowledge management; 2011. p. 1047–1052.
3.
Zurück zum Zitat Pan SJ, Tsang IW, Kwok JT, Yang Q. Domain adaptation via transfer component analysis. IEEE Trans Neural Netw. 2011;22(2):199–210.CrossRefPubMed Pan SJ, Tsang IW, Kwok JT, Yang Q. Domain adaptation via transfer component analysis. IEEE Trans Neural Netw. 2011;22(2):199–210.CrossRefPubMed
4.
Zurück zum Zitat Patel VM, Gopalan R, Li R, Chellappa R. Visual domain adaptation: a survey of recent advances. IEEE Signal Proc Mag. 2015;32(3):53–69.CrossRef Patel VM, Gopalan R, Li R, Chellappa R. Visual domain adaptation: a survey of recent advances. IEEE Signal Proc Mag. 2015;32(3):53–69.CrossRef
5.
Zurück zum Zitat Gong B, Shi Y, Sha F, Grauman K. Geodesic flow kernel for unsupervised domain adaptation. IEEE conference on computer vision and pattern recognition (CVPR); 2012. p. 2066–2073. Gong B, Shi Y, Sha F, Grauman K. Geodesic flow kernel for unsupervised domain adaptation. IEEE conference on computer vision and pattern recognition (CVPR); 2012. p. 2066–2073.
6.
Zurück zum Zitat Yu J, Hong C, Rui Y, Tao D. Multi-task autoencoder model for recovering human poses. IEEE Trans Ind Electron. 2017;65(6):5060–5068.CrossRef Yu J, Hong C, Rui Y, Tao D. Multi-task autoencoder model for recovering human poses. IEEE Trans Ind Electron. 2017;65(6):5060–5068.CrossRef
7.
Zurück zum Zitat Ando S, Suzuki E. Unsupervised cross-domain learning by interaction information co-clustering. Eighth IEEE international conference on data mining data mining; 2008. p. 13–22. Ando S, Suzuki E. Unsupervised cross-domain learning by interaction information co-clustering. Eighth IEEE international conference on data mining data mining; 2008. p. 13–22.
8.
Zurück zum Zitat Zhang L, Zhang D. Robust visual knowledge transfer via extreme learning machine-based domain adaptation. IEEE Trans Image Process. 2016;25(10):4959–4973.CrossRefPubMed Zhang L, Zhang D. Robust visual knowledge transfer via extreme learning machine-based domain adaptation. IEEE Trans Image Process. 2016;25(10):4959–4973.CrossRefPubMed
9.
Zurück zum Zitat Zhang L, Zhang D. Domain adaptation extreme learning machines for drift compensation in E-nose systems. IEEE Trans Instrum Meas. 2015;64(7):1790–1801.CrossRef Zhang L, Zhang D. Domain adaptation extreme learning machines for drift compensation in E-nose systems. IEEE Trans Instrum Meas. 2015;64(7):1790–1801.CrossRef
10.
Zurück zum Zitat Pan SJ, Kwok JT, Yang Q. Transfer learning via dimensionality reduction. Proceedings of the Twenty-Third AAAI conference on artificial intelligence; 2008. p. 677–682. Pan SJ, Kwok JT, Yang Q. Transfer learning via dimensionality reduction. Proceedings of the Twenty-Third AAAI conference on artificial intelligence; 2008. p. 677–682.
11.
Zurück zum Zitat Borgwardt KM, Gretton A, Rasch MJ, Kriegel HP, Schölkopf B, Smola AJ. Integrating structured biological data by kernel maximum mean discrepancy. Bioinformatics. 2006; 22(14):e49–e57. Borgwardt KM, Gretton A, Rasch MJ, Kriegel HP, Schölkopf B, Smola AJ. Integrating structured biological data by kernel maximum mean discrepancy. Bioinformatics. 2006; 22(14):e49–e57.
12.
Zurück zum Zitat Jiang M, Huang W, Huang Z, Yen GG. Integration of global and local metrics for domain adaptation learning via dimensionality reduction. IEEE Trans Cybern. 2017;47(1):38–51.CrossRefPubMed Jiang M, Huang W, Huang Z, Yen GG. Integration of global and local metrics for domain adaptation learning via dimensionality reduction. IEEE Trans Cybern. 2017;47(1):38–51.CrossRefPubMed
13.
Zurück zum Zitat Long M, Cao Y, Wang J, Jordan MI. Learning transferable features with deep adaptation networks. International conference on machine learning; 2015. p. 97–105. Long M, Cao Y, Wang J, Jordan MI. Learning transferable features with deep adaptation networks. International conference on machine learning; 2015. p. 97–105.
14.
Zurück zum Zitat Fernando B, Habrard A, Sebban M, Tuytelaars T. Unsupervised visual domain adaptation using subspace alignment. Proceedings of the IEEE international conference on computer vision (ICCV); 2013. p. 2960–2967. Fernando B, Habrard A, Sebban M, Tuytelaars T. Unsupervised visual domain adaptation using subspace alignment. Proceedings of the IEEE international conference on computer vision (ICCV); 2013. p. 2960–2967.
16.
Zurück zum Zitat Hong C, Zeng Z, Xie R, Zhuang W, Wang X. Domain adaptation with low-rank alignment for weakly supervised hand pose recovery. Signal Process. 2018;142:223–230.CrossRef Hong C, Zeng Z, Xie R, Zhuang W, Wang X. Domain adaptation with low-rank alignment for weakly supervised hand pose recovery. Signal Process. 2018;142:223–230.CrossRef
17.
Zurück zum Zitat Long M, Wang J, Ding G, Sun J, Yu PS. Transfer feature learning with joint distribution adaptation. Proceedings of the IEEE international conference on computer vision (ICCV); 2013. p. 2200–2207. Long M, Wang J, Ding G, Sun J, Yu PS. Transfer feature learning with joint distribution adaptation. Proceedings of the IEEE international conference on computer vision (ICCV); 2013. p. 2200–2207.
18.
Zurück zum Zitat Tahmoresnezhad J, Hashemi S. Visual domain adaptation via transfer feature learning. Knowl Inf Syst 2017; 50(2):585–605.CrossRef Tahmoresnezhad J, Hashemi S. Visual domain adaptation via transfer feature learning. Knowl Inf Syst 2017; 50(2):585–605.CrossRef
19.
Zurück zum Zitat Luo L, Wang X, Hu S, Chen L. Robust data geometric structure aligned close yet discriminative domain adaptation. 2017. arXiv:1705.08620v1. Luo L, Wang X, Hu S, Chen L. Robust data geometric structure aligned close yet discriminative domain adaptation. 2017. arXiv:1705.​08620v1.
20.
Zurück zum Zitat Long M, Wang J, Ding G, Pan SJ, Yu PS. Adaptation regularization: a general framework for transfer learning. IEEE Trans Knowl Data Eng. 2014;26(5):1076–1089.CrossRef Long M, Wang J, Ding G, Pan SJ, Yu PS. Adaptation regularization: a general framework for transfer learning. IEEE Trans Knowl Data Eng. 2014;26(5):1076–1089.CrossRef
21.
Zurück zum Zitat Gheisari M, Baghshah MS. Joint predictive model and representation learning for visual domain adaptation. Eng Appl Artif Intel 2017;58:157–170.CrossRef Gheisari M, Baghshah MS. Joint predictive model and representation learning for visual domain adaptation. Eng Appl Artif Intel 2017;58:157–170.CrossRef
23.
Zurück zum Zitat Lu B, Chellappa R, Nasrabadi NM. Incremental dictionary learning for unsupervised domain adaptation. British machine vision conference (BMVC); 2015. p. 108.1–108.12. Lu B, Chellappa R, Nasrabadi NM. Incremental dictionary learning for unsupervised domain adaptation. British machine vision conference (BMVC); 2015. p. 108.1–108.12.
24.
Zurück zum Zitat Hou CA, Yeh YR, Wang YCF. An unsupervised domain adaptation approach for cross-domain visual classification. IEEE international conference on advanced video and signal based surveillance (AVSS); 2015. p. 1–6. Hou CA, Yeh YR, Wang YCF. An unsupervised domain adaptation approach for cross-domain visual classification. IEEE international conference on advanced video and signal based surveillance (AVSS); 2015. p. 1–6.
25.
Zurück zum Zitat Huang GB, Zhu QY, Siew CK. Extreme learning machine: theory and applications. Neurocomputing. 2006; 70(1):489–501.CrossRef Huang GB, Zhu QY, Siew CK. Extreme learning machine: theory and applications. Neurocomputing. 2006; 70(1):489–501.CrossRef
26.
Zurück zum Zitat Huang GB, Zhou H, Ding X, Zhang R. Extreme learning machine for regression and multiclass classification. IEEE Trans Syst Man Cybern B Cybern. 2012;42(2):513–529.CrossRefPubMed Huang GB, Zhou H, Ding X, Zhang R. Extreme learning machine for regression and multiclass classification. IEEE Trans Syst Man Cybern B Cybern. 2012;42(2):513–529.CrossRefPubMed
27.
Zurück zum Zitat Huang GB. What are extreme learning machines? Filling the gap between Frank Rosenblatts dream and John von Neumanns puzzle. Cogn Comput 2015;7(3):263–278.CrossRef Huang GB. What are extreme learning machines? Filling the gap between Frank Rosenblatts dream and John von Neumanns puzzle. Cogn Comput 2015;7(3):263–278.CrossRef
28.
Zurück zum Zitat Guo T, Zhang L, Tan X. Neuron pruning-based discriminative extreme learning machine for pattern classification. Cogn Comput. 2017;9(4):581–595.CrossRef Guo T, Zhang L, Tan X. Neuron pruning-based discriminative extreme learning machine for pattern classification. Cogn Comput. 2017;9(4):581–595.CrossRef
29.
Zurück zum Zitat Liu Y, Zhang L, Deng P, He Z. Common subspace learning via cross-domain extreme learning machine. Cogn Comput. 2017;9(4):555–563.CrossRef Liu Y, Zhang L, Deng P, He Z. Common subspace learning via cross-domain extreme learning machine. Cogn Comput. 2017;9(4):555–563.CrossRef
30.
Zurück zum Zitat Peng Y, Wang S, Long X, Lu B. Discriminative graph regularized extreme learning machine and its application to face recognition. Neurocomputing. 2015;149:340–353.CrossRef Peng Y, Wang S, Long X, Lu B. Discriminative graph regularized extreme learning machine and its application to face recognition. Neurocomputing. 2015;149:340–353.CrossRef
31.
Zurück zum Zitat Huang G, Song S, Gupta JND, Wu C. Semi-supervised and unsupervised extreme learning machines. IEEE Trans Cybern 2014;44(12):2405–2417.CrossRefPubMed Huang G, Song S, Gupta JND, Wu C. Semi-supervised and unsupervised extreme learning machines. IEEE Trans Cybern 2014;44(12):2405–2417.CrossRefPubMed
32.
Zurück zum Zitat Iosifidis A, Tefas A, Pitas I. Graph embedded extreme learning machine. IEEE Trans Cybern 2016;46 (1):311–324.CrossRefPubMed Iosifidis A, Tefas A, Pitas I. Graph embedded extreme learning machine. IEEE Trans Cybern 2016;46 (1):311–324.CrossRefPubMed
34.
Zurück zum Zitat Iosifidis A, Tefas A, Pitas I. Semi-supervised classification of human actions based on neural networks. 22nd international conference on pattern recognition (ICPR), IEEE; 2014. p. 1336–1341. Iosifidis A, Tefas A, Pitas I. Semi-supervised classification of human actions based on neural networks. 22nd international conference on pattern recognition (ICPR), IEEE; 2014. p. 1336–1341.
35.
Zurück zum Zitat Xiang S, Nie F, Meng G, Pan C, Zhang C. Discriminative least squares regression for multiclass classification and feature selection. IEEE Trans Neural Netw Learn Syst. 2012;23(11):1738–1754.CrossRefPubMed Xiang S, Nie F, Meng G, Pan C, Zhang C. Discriminative least squares regression for multiclass classification and feature selection. IEEE Trans Neural Netw Learn Syst. 2012;23(11):1738–1754.CrossRefPubMed
36.
Zurück zum Zitat Xu Y, Fang X, Wu J, Li X, Zhang D. Discriminative transfer subspace learning via low-rank and sparse representation. IEEE Trans Image Process. 2016;25(2):850–863.CrossRefPubMed Xu Y, Fang X, Wu J, Li X, Zhang D. Discriminative transfer subspace learning via low-rank and sparse representation. IEEE Trans Image Process. 2016;25(2):850–863.CrossRefPubMed
37.
Zurück zum Zitat Lei B, Yang P, Wang T, Chen S, Ni D. Relational-Regularized Discriminative sparse learning for alzheimers disease diagnosis. IEEE Trans Cybern. 2017;47(4):1102–1113.CrossRefPubMed Lei B, Yang P, Wang T, Chen S, Ni D. Relational-Regularized Discriminative sparse learning for alzheimers disease diagnosis. IEEE Trans Cybern. 2017;47(4):1102–1113.CrossRefPubMed
38.
Zurück zum Zitat Luo M, Zhang K. A hybrid approach combining extreme learning machine and sparse representation for image classification. Eng Appl Artif Intel 2014;27:228–235.CrossRef Luo M, Zhang K. A hybrid approach combining extreme learning machine and sparse representation for image classification. Eng Appl Artif Intel 2014;27:228–235.CrossRef
39.
Zurück zum Zitat Li X, Mao W, Jiang W. Extreme learning machine based transfer learning for data classification. Neurocomputing. 2016;174:203–210.CrossRef Li X, Mao W, Jiang W. Extreme learning machine based transfer learning for data classification. Neurocomputing. 2016;174:203–210.CrossRef
40.
Zurück zum Zitat Zhang L, He Z, Liu Y. Deep object recognition across domains based on adaptive extreme learning machine. Neurocomputing. 2017;239:194–203.CrossRef Zhang L, He Z, Liu Y. Deep object recognition across domains based on adaptive extreme learning machine. Neurocomputing. 2017;239:194–203.CrossRef
41.
Zurück zum Zitat Uzair M, Mian A. Blind domain adaptation with augmented extreme learning machine features. IEEE Trans Cybern. 2017;47(3):651–660.CrossRefPubMed Uzair M, Mian A. Blind domain adaptation with augmented extreme learning machine features. IEEE Trans Cybern. 2017;47(3):651–660.CrossRefPubMed
42.
Zurück zum Zitat Wang L, Zhang XY, Pan C. MSDLSR Margin Scalable discriminative least squares regression for multicategory classification. IEEE Trans Neural Netw Learn Syst. 2016;27(12):2711–2717.CrossRefPubMed Wang L, Zhang XY, Pan C. MSDLSR Margin Scalable discriminative least squares regression for multicategory classification. IEEE Trans Neural Netw Learn Syst. 2016;27(12):2711–2717.CrossRefPubMed
43.
Zurück zum Zitat Pan J, Wang X, Cheng Y, Cao G. Multi-source transfer ELM-based Q learning. Neurocomputing 2014; 137:57–64.CrossRef Pan J, Wang X, Cheng Y, Cao G. Multi-source transfer ELM-based Q learning. Neurocomputing 2014; 137:57–64.CrossRef
44.
Zurück zum Zitat Cortes C, Vapnik V. Support-vector networks. Mach Learn. 1995;20(3):273–297. Cortes C, Vapnik V. Support-vector networks. Mach Learn. 1995;20(3):273–297.
45.
Zurück zum Zitat Lu Y, Lai Z, Fan Z, Cui J, Zhu Q. Manifold discriminant regression learning for image classification. Neurocomputing 2015;166:475–486.CrossRef Lu Y, Lai Z, Fan Z, Cui J, Zhu Q. Manifold discriminant regression learning for image classification. Neurocomputing 2015;166:475–486.CrossRef
46.
Zurück zum Zitat Al-Shedivat M, Wang JJY, Alzahrani M, Huang JZ, Gao X. Supervised transfer sparse coding. Proceedings of the Twenty-Eighth AAAI conference on artificial intelligence; 2014. p. 1665–1672. Al-Shedivat M, Wang JJY, Alzahrani M, Huang JZ, Gao X. Supervised transfer sparse coding. Proceedings of the Twenty-Eighth AAAI conference on artificial intelligence; 2014. p. 1665–1672.
47.
Zurück zum Zitat Iosifidis A, Mygdalis V, Tefas A, Pitas I. One-class classification based on extreme learning and geometric class information. Neural Process Lett. 2017;45(2):577–592.CrossRef Iosifidis A, Mygdalis V, Tefas A, Pitas I. One-class classification based on extreme learning and geometric class information. Neural Process Lett. 2017;45(2):577–592.CrossRef
48.
Zurück zum Zitat Saenko K, Kulis B, Fritz M, Darrell T. Adapting visual category models to new domains. European conference on computer vision (ECCV); 2010. p. 213–226. Saenko K, Kulis B, Fritz M, Darrell T. Adapting visual category models to new domains. European conference on computer vision (ECCV); 2010. p. 213–226.
49.
Zurück zum Zitat Griffin G, Holub AD, Perona P. Caltech-256 object category dataset. Caltech Technical Report. 2007. Griffin G, Holub AD, Perona P. Caltech-256 object category dataset. Caltech Technical Report. 2007.
50.
Zurück zum Zitat LeCun Y, Bottou L, Bengio Y, Haffner P. Gradient-based learning applied to document recognition. Proc IEEE 1998;86(11):2278–2324.CrossRef LeCun Y, Bottou L, Bengio Y, Haffner P. Gradient-based learning applied to document recognition. Proc IEEE 1998;86(11):2278–2324.CrossRef
51.
Zurück zum Zitat Hull JJ. A database for handwritten text recognition research. IEEE Trans Pattern Anal Mach Intell 1994;16 (5):550–554.CrossRef Hull JJ. A database for handwritten text recognition research. IEEE Trans Pattern Anal Mach Intell 1994;16 (5):550–554.CrossRef
52.
Zurück zum Zitat Nene SA, Nayar SK, Murase H. 1996. Columbia object image library (COIL-20). Technical Report CUCS-005-96. Nene SA, Nayar SK, Murase H. 1996. Columbia object image library (COIL-20). Technical Report CUCS-005-96.
53.
Zurück zum Zitat Chang CC, Lin CJ. LIBSVM: A library for support vector machines. ACM TIST. 2011;2(3):27. Chang CC, Lin CJ. LIBSVM: A library for support vector machines. ACM TIST. 2011;2(3):27.
54.
Zurück zum Zitat Ghifary M, Kleijn WB, Zhang M. Domain adaptive neural networks for object recognition. Pacific Rim international conference on artificial intelligence; 2014. p. 898–904. Ghifary M, Kleijn WB, Zhang M. Domain adaptive neural networks for object recognition. Pacific Rim international conference on artificial intelligence; 2014. p. 898–904.
55.
Zurück zum Zitat Yu Y, Sun Z. Sparse coding extreme learning machine for classification. Neurocomputing 2017;261:50–56.CrossRef Yu Y, Sun Z. Sparse coding extreme learning machine for classification. Neurocomputing 2017;261:50–56.CrossRef
57.
Zurück zum Zitat Cao J, Zhang K, Luo M, Yin C, Lai X. Extreme learning machine and adaptive sparse representation for image classification. Neural Networks. 2016;81(C):91–102.CrossRefPubMed Cao J, Zhang K, Luo M, Yin C, Lai X. Extreme learning machine and adaptive sparse representation for image classification. Neural Networks. 2016;81(C):91–102.CrossRefPubMed
58.
Zurück zum Zitat Chen Y, Song S. Domain transfer extreme learning machine and its application on domain adaptation problems. China Sciencepaper. 2017;12(14):1565–1569 + 1609. Chen Y, Song S. Domain transfer extreme learning machine and its application on domain adaptation problems. China Sciencepaper. 2017;12(14):1565–1569 + 1609.
Metadaten
Titel
A Joint Unsupervised Cross-Domain Model via Scalable Discriminative Extreme Learning Machine
verfasst von
Boyang Zhang
Yingyi Liu
Haiwen Yuan
Lingjie Sun
Zhao Ma
Publikationsdatum
21.04.2018
Verlag
Springer US
Erschienen in
Cognitive Computation / Ausgabe 4/2018
Print ISSN: 1866-9956
Elektronische ISSN: 1866-9964
DOI
https://doi.org/10.1007/s12559-018-9555-z

Weitere Artikel der Ausgabe 4/2018

Cognitive Computation 4/2018 Zur Ausgabe