Skip to main content
Erschienen in: Neural Processing Letters 2/2018

19.12.2017

Domain Adaptation with Twin Support Vector Machines

verfasst von: Xijiong Xie, Shiliang Sun, Huahui Chen, Jiangbo Qian

Erschienen in: Neural Processing Letters | Ausgabe 2/2018

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Supervised learning models assume that the training and test data are drawn from the same underlying distribution. These algorithms become ineffective when such assumptions are violated. Therefore, domain adaptation methods were proposed to handle data from different, yet correlated distributions. They have been applied in many fields such as computer vision, sentiment analysis, natural language processing and so on. Often in these cases, source domains provide large numbers of labeled data for learning, while target domain may have few labeled data available. In this paper, we explore a new supervised domain adaption method with twin support vector machines called aggregate domain adaptation twin support vector machines, which obtain the classifiers trained on the source and target training data. Then we evaluate the performance of our proposed methods on brain-computer interfaces and sentiment analysis datasets. The experimental results show the effectiveness and the reliability of our proposed methods.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Pan S, Yang Q (2010) A survey on transfer learning. IEEE Trans Knowl Data Eng 22:1345–1359CrossRef Pan S, Yang Q (2010) A survey on transfer learning. IEEE Trans Knowl Data Eng 22:1345–1359CrossRef
2.
Zurück zum Zitat Sun S, Shi H, Wu Y (2015) A survey of multi-source domain adaptation. Inf Fusion 24:84–92CrossRef Sun S, Shi H, Wu Y (2015) A survey of multi-source domain adaptation. Inf Fusion 24:84–92CrossRef
3.
Zurück zum Zitat Klinkenberg R, Joachims T (2000) Detecting concept drift with support vector machines. In: Proceedings of the international conference on machine learning, pp 487–494 Klinkenberg R, Joachims T (2000) Detecting concept drift with support vector machines. In: Proceedings of the international conference on machine learning, pp 487–494
4.
Zurück zum Zitat Wu P, Dietterich T (2004) Improving SVM accuracy by training on auxiliary data sources. In: Proceedings of the international conference on machine learning, pp 871–878 Wu P, Dietterich T (2004) Improving SVM accuracy by training on auxiliary data sources. In: Proceedings of the international conference on machine learning, pp 871–878
5.
Zurück zum Zitat Liao X, Xue Y, Carin L (2005) Logistic regression with an auxiliary data source. In: Proceedings of the 21st international conference on machine learning, pp 505–512 Liao X, Xue Y, Carin L (2005) Logistic regression with an auxiliary data source. In: Proceedings of the 21st international conference on machine learning, pp 505–512
6.
7.
Zurück zum Zitat Blitzer J, Dredze M, Pereira F (2007) Boom-boxes and blenders: domain adaptation for sentiment classification. In: Proceedings of the 45th annual meeting of the assocation for computational linguistics, pp 440–447 Blitzer J, Dredze M, Pereira F (2007) Boom-boxes and blenders: domain adaptation for sentiment classification. In: Proceedings of the 45th annual meeting of the assocation for computational linguistics, pp 440–447
8.
Zurück zum Zitat Ling X, Dai W, Xue G, Yang Q, Yu Y (2008) Spectral domain transfer learning. In: Proceedings of the 14th ACM SIGKDD international conference on knowledge discovery and data mining, pp 488–496 Ling X, Dai W, Xue G, Yang Q, Yu Y (2008) Spectral domain transfer learning. In: Proceedings of the 14th ACM SIGKDD international conference on knowledge discovery and data mining, pp 488–496
9.
Zurück zum Zitat Pan S, Tsang I, Kwok J, Yang Q (2011) Domain adaptation via transfer component analysis. IEEE Trans Neural Netw 22:199–210CrossRef Pan S, Tsang I, Kwok J, Yang Q (2011) Domain adaptation via transfer component analysis. IEEE Trans Neural Netw 22:199–210CrossRef
10.
Zurück zum Zitat Satpal S, Sarawagi S (2007) Domain adaptation of conditional probability models via feature subsetting. In: Proceedings of the 11th European conference on principles and practice of knowledge discovery in databases, pp 224–235 Satpal S, Sarawagi S (2007) Domain adaptation of conditional probability models via feature subsetting. In: Proceedings of the 11th European conference on principles and practice of knowledge discovery in databases, pp 224–235
11.
Zurück zum Zitat Seah C, Tsang I, Ong Y, Lee K (2010) Predictive distribution matching SVM for multi-domain learning. In: Proceedings of the European conference on machine learning and principles and practice of knowledge discovery in databases, pp 231–247CrossRef Seah C, Tsang I, Ong Y, Lee K (2010) Predictive distribution matching SVM for multi-domain learning. In: Proceedings of the European conference on machine learning and principles and practice of knowledge discovery in databases, pp 231–247CrossRef
12.
Zurück zum Zitat Pan S, Kwok J, Yang Q (2008) Transfer learning via dimensionality reduction. In: Proceedings of the association for the advancement of artificial intelligence, pp 677–682 Pan S, Kwok J, Yang Q (2008) Transfer learning via dimensionality reduction. In: Proceedings of the association for the advancement of artificial intelligence, pp 677–682
13.
Zurück zum Zitat Zhong E, Fan W, Peng J, Zhang K, Ren J, Turaga D, Verscheure O (2009) Cross domain distribution adaptation via kernel mapping. In: Proceedings of the ACM SIGKDD international conference on knowledge discovery and data mining, pp 1027–1036 Zhong E, Fan W, Peng J, Zhang K, Ren J, Turaga D, Verscheure O (2009) Cross domain distribution adaptation via kernel mapping. In: Proceedings of the ACM SIGKDD international conference on knowledge discovery and data mining, pp 1027–1036
14.
Zurück zum Zitat Shawe-Taylor J, Sun S (2011) A review of optimization methodologies in support vector machines. Neurocomputing 74:3609–3618CrossRef Shawe-Taylor J, Sun S (2011) A review of optimization methodologies in support vector machines. Neurocomputing 74:3609–3618CrossRef
15.
Zurück zum Zitat Li J, Xu C, Yang W, Sun C, Tao D (2015) Discriminative multi-view interactive image re-ranking. IEEE Trans Image Process 26:3113–3127MathSciNetCrossRef Li J, Xu C, Yang W, Sun C, Tao D (2015) Discriminative multi-view interactive image re-ranking. IEEE Trans Image Process 26:3113–3127MathSciNetCrossRef
16.
Zurück zum Zitat Xu C, Tao D, Xu C (2015) Large-margin multi-label causal feature learning. In: Proceedings of the AAAI conference on artificial intelligence, pp 1924–1930 Xu C, Tao D, Xu C (2015) Large-margin multi-label causal feature learning. In: Proceedings of the AAAI conference on artificial intelligence, pp 1924–1930
17.
Zurück zum Zitat Xu C, Tao D, Li Y, Xu C (2013) Large-margin multi-view Gaussian process for image classification. In: Proceedings of the international conference on internet multimedia computing and service, pp 7–12 Xu C, Tao D, Li Y, Xu C (2013) Large-margin multi-view Gaussian process for image classification. In: Proceedings of the international conference on internet multimedia computing and service, pp 7–12
18.
Zurück zum Zitat Blitzer J, McDonald R, Pereira F (2006) Domain adaptation with structural correspondence learning. In: Proceedings of the conference on empiriacal methods in natural language processing, pp 120–128 Blitzer J, McDonald R, Pereira F (2006) Domain adaptation with structural correspondence learning. In: Proceedings of the conference on empiriacal methods in natural language processing, pp 120–128
19.
Zurück zum Zitat Daum\(\acute{e}\) III H (2007) Frustratingly easy domain adaptation. In: Proceedings of the 45th annual meeting of the assocation for computational linguistics, pp 256–263 Daum\(\acute{e}\) III H (2007) Frustratingly easy domain adaptation. In: Proceedings of the 45th annual meeting of the assocation for computational linguistics, pp 256–263
20.
Zurück zum Zitat Yang J, Yan R, Hauptmann A (2007) Cross-domain video concept detection using adaptive svms. In: Proceedings of the international conference on multimedia, pp 188–197 Yang J, Yan R, Hauptmann A (2007) Cross-domain video concept detection using adaptive svms. In: Proceedings of the international conference on multimedia, pp 188–197
21.
Zurück zum Zitat Jiang W, Zavesky E, Chang S, Loui A (2008) Cross-domain learning methods for high-level visual concept classification. In: Proceedings of the international conference on image processing, pp 161–164 Jiang W, Zavesky E, Chang S, Loui A (2008) Cross-domain learning methods for high-level visual concept classification. In: Proceedings of the international conference on image processing, pp 161–164
22.
Zurück zum Zitat Storkey A, Sugiyama M (2006) Mixture regression for covariate shift. In: Proceedings of the conference on advances in neural information processing systems, pp 1337–1344 Storkey A, Sugiyama M (2006) Mixture regression for covariate shift. In: Proceedings of the conference on advances in neural information processing systems, pp 1337–1344
23.
Zurück zum Zitat Huang J, Sola A, Gretton A, Borgwardt K, Schölkopf B (2006) Correcting sample selection bias by unlabeled data. Adv Neural Inf Proces Syst 19:601–608 Huang J, Sola A, Gretton A, Borgwardt K, Schölkopf B (2006) Correcting sample selection bias by unlabeled data. Adv Neural Inf Proces Syst 19:601–608
24.
Zurück zum Zitat Crammer K, Kearns M, Wortman J (2008) Learning from multiple sources. J Mach Learn Res 9:1757–1774MathSciNetMATH Crammer K, Kearns M, Wortman J (2008) Learning from multiple sources. J Mach Learn Res 9:1757–1774MathSciNetMATH
25.
Zurück zum Zitat Luo P, Zhuang F, Xiong H, Xiong Y, He Q (2008) Transfer learning from multiple source domains via consensus regularization. In: Proceedings of the ACM conference on information and knowledge management, pp 103–112 Luo P, Zhuang F, Xiong H, Xiong Y, He Q (2008) Transfer learning from multiple source domains via consensus regularization. In: Proceedings of the ACM conference on information and knowledge management, pp 103–112
26.
Zurück zum Zitat Mansour Y, Mohri M, Rostamizadeh A (2009) Domain adaptation: learning bounds and algorithms. In: Proceedings of the 22nd annual conference on learning theory, pp 34–47 Mansour Y, Mohri M, Rostamizadeh A (2009) Domain adaptation: learning bounds and algorithms. In: Proceedings of the 22nd annual conference on learning theory, pp 34–47
27.
Zurück zum Zitat Schweikert G, Widmer C, Schölkopf B, Rätsch G (2008) An empirical analysis of domain adaptation algorithm for genomic sequence analysis. Adv Neural Inf Process Syst 21:1433–1440 Schweikert G, Widmer C, Schölkopf B, Rätsch G (2008) An empirical analysis of domain adaptation algorithm for genomic sequence analysis. Adv Neural Inf Process Syst 21:1433–1440
28.
Zurück zum Zitat Blitzer J, Crammer K, Kulesza A, Pereira F, Wortman J (2007) Learning bounds for domain adaptation. In: Proceedings of conference on advances in neural information processing systems, pp 129–136 Blitzer J, Crammer K, Kulesza A, Pereira F, Wortman J (2007) Learning bounds for domain adaptation. In: Proceedings of conference on advances in neural information processing systems, pp 129–136
29.
Zurück zum Zitat Duan L, Tsang I, Xu D, Maybank S (2009) Domain transfer SVM for video concept detection. In: Proceedings of the international conference computer vision and pattern recognition, pp 1375–1381 Duan L, Tsang I, Xu D, Maybank S (2009) Domain transfer SVM for video concept detection. In: Proceedings of the international conference computer vision and pattern recognition, pp 1375–1381
30.
Zurück zum Zitat Duan L, Tsang I, Xu D (2012) Domain transfer multiple kernel learning. IEEE Trans Pattern Anal Mach Intell 34:465–479CrossRef Duan L, Tsang I, Xu D (2012) Domain transfer multiple kernel learning. IEEE Trans Pattern Anal Mach Intell 34:465–479CrossRef
31.
Zurück zum Zitat Duan L, Xu D, Tsang I (2012) Domain adaptation from multiple sources: a domain-dependent regularization approach. IEEE Trans Neural Netw Learn Syst 23:504–518CrossRef Duan L, Xu D, Tsang I (2012) Domain adaptation from multiple sources: a domain-dependent regularization approach. IEEE Trans Neural Netw Learn Syst 23:504–518CrossRef
32.
Zurück zum Zitat Bruzzone L, Marconcini M (2010) Domain adaptation problems: a DASVM classification techique and a circular validation strategy. IEEE Trans Pattern Anal Mach Intell 32:770–787CrossRef Bruzzone L, Marconcini M (2010) Domain adaptation problems: a DASVM classification techique and a circular validation strategy. IEEE Trans Pattern Anal Mach Intell 32:770–787CrossRef
33.
Zurück zum Zitat Tao J, Chung F, Wang S (2012) On minimum distribution discrepancy support vector machine for domain adaptation. Pattern Recognit 45:3962–3984CrossRef Tao J, Chung F, Wang S (2012) On minimum distribution discrepancy support vector machine for domain adaptation. Pattern Recognit 45:3962–3984CrossRef
34.
Zurück zum Zitat Venkateswara H, Lade P, Ye J, Panchanathan S (2015) Coupled support vector machines for supervised domain adaptation. In: Proceedings of the ACM international conference on multimedia, pp 1295–1298 Venkateswara H, Lade P, Ye J, Panchanathan S (2015) Coupled support vector machines for supervised domain adaptation. In: Proceedings of the ACM international conference on multimedia, pp 1295–1298
35.
Zurück zum Zitat Gong M, Zhang K, Liu T, Tao D, Glymour C, Schlkopf B (2016) Domain adaptation with conditional transferable components. In: Proceedings of the 33rd international conference on machine learning, pp 2839–2848 Gong M, Zhang K, Liu T, Tao D, Glymour C, Schlkopf B (2016) Domain adaptation with conditional transferable components. In: Proceedings of the 33rd international conference on machine learning, pp 2839–2848
36.
37.
Zurück zum Zitat Mangasarian O, Wild E (2006) MultisurFace proximal support vector machine classification via generalized eigenvalues. IEEE Trans Pattern Anal Mach Intell 28:69–74CrossRef Mangasarian O, Wild E (2006) MultisurFace proximal support vector machine classification via generalized eigenvalues. IEEE Trans Pattern Anal Mach Intell 28:69–74CrossRef
38.
Zurück zum Zitat Jayadeva R, Khemchandani S, Chandra (2007) Twin support vector machines for pattern classification. IEEE Trans on Pattern Anal Mach Intell 74:905–910CrossRef Jayadeva R, Khemchandani S, Chandra (2007) Twin support vector machines for pattern classification. IEEE Trans on Pattern Anal Mach Intell 74:905–910CrossRef
39.
Zurück zum Zitat Shao Y, Zhang C, Wang X, Deng N (2011) Improvements on twin support vector machines. IEEE Trans Neural Netw 22:962–968CrossRef Shao Y, Zhang C, Wang X, Deng N (2011) Improvements on twin support vector machines. IEEE Trans Neural Netw 22:962–968CrossRef
40.
Zurück zum Zitat Ding S, Zhao Y, Qi B, Huang H (2014) An overview on twin support vector machines. Artif Intell Rev 42:245–252CrossRef Ding S, Zhao Y, Qi B, Huang H (2014) An overview on twin support vector machines. Artif Intell Rev 42:245–252CrossRef
41.
Zurück zum Zitat Kumar M, Gopal M (2009) Least squares twin support vector machines for pattern classification. Expert Syst Appl 36:7535–7543CrossRef Kumar M, Gopal M (2009) Least squares twin support vector machines for pattern classification. Expert Syst Appl 36:7535–7543CrossRef
42.
Zurück zum Zitat Xu Y, Pan X, Zhou Z, Yang Z, Zhang Y (2015) Structural least square twin support vector machine for classification. Appl Intell 42:1–10CrossRef Xu Y, Pan X, Zhou Z, Yang Z, Zhang Y (2015) Structural least square twin support vector machine for classification. Appl Intell 42:1–10CrossRef
43.
Zurück zum Zitat Xie X, Sun S (2015) Multitask centroid twin support vector machines. Neurocomputing 149:1085–1091CrossRef Xie X, Sun S (2015) Multitask centroid twin support vector machines. Neurocomputing 149:1085–1091CrossRef
44.
Zurück zum Zitat Xie X, Sun S (2015) Multi-view twin support vector machines. Intell Data Anal 19:701–712CrossRef Xie X, Sun S (2015) Multi-view twin support vector machines. Intell Data Anal 19:701–712CrossRef
45.
Zurück zum Zitat Xie X, Sun S (2014) Multi-view Laplacian twin support vector machines. Appl Intell 41:1059–1068CrossRef Xie X, Sun S (2014) Multi-view Laplacian twin support vector machines. Appl Intell 41:1059–1068CrossRef
46.
Zurück zum Zitat Chen W, Shao Y, Li C, Deng N (2016) MLTSVM: a novel twin support vector machine to multi-label learning. Pattern Recognit 52:61–74CrossRef Chen W, Shao Y, Li C, Deng N (2016) MLTSVM: a novel twin support vector machine to multi-label learning. Pattern Recognit 52:61–74CrossRef
47.
Zurück zum Zitat Qi Z, Tian Y, Shi Y (2012) Laplacian twin support vector machine for semi-supervised classification. Neural Netw 35:46–53CrossRef Qi Z, Tian Y, Shi Y (2012) Laplacian twin support vector machine for semi-supervised classification. Neural Netw 35:46–53CrossRef
48.
Zurück zum Zitat Peng X (2010) An efficient twin support vector machine for regression. Neural Netw 23:365–372CrossRef Peng X (2010) An efficient twin support vector machine for regression. Neural Netw 23:365–372CrossRef
49.
Zurück zum Zitat Xie J, Hone K, Xie W, Gao X, Shi Y, Liu X (2013) Extending twin support vector machine classifier for multi-category classification problems. Intell Data Anal 17:649–664 Xie J, Hone K, Xie W, Gao X, Shi Y, Liu X (2013) Extending twin support vector machine classifier for multi-category classification problems. Intell Data Anal 17:649–664
50.
Zurück zum Zitat Zhang K, Schlkopf B, Muandet K, Wang Z (2013) Domain adaptation under target and conditional shift. J Volcanol Geotherm Res 173:185–195 Zhang K, Schlkopf B, Muandet K, Wang Z (2013) Domain adaptation under target and conditional shift. J Volcanol Geotherm Res 173:185–195
51.
Zurück zum Zitat Liu T, Yang Q, Tao D (2017) Understanding how feature structure transfers in transfer learning. In: Proceedings of the twenty-sixth international joint conference on artificial intelligence, pp 2365–2371 Liu T, Yang Q, Tao D (2017) Understanding how feature structure transfers in transfer learning. In: Proceedings of the twenty-sixth international joint conference on artificial intelligence, pp 2365–2371
52.
Zurück zum Zitat Hong C, Yu J, Wan J, Tao D, Wang M (2015) Multimodal deep autoencoder for human pose recovery. IEEE Trans Image Process 24:56–59MathSciNetCrossRef Hong C, Yu J, Wan J, Tao D, Wang M (2015) Multimodal deep autoencoder for human pose recovery. IEEE Trans Image Process 24:56–59MathSciNetCrossRef
53.
Zurück zum Zitat Yu J, Zhang B, Kuang Z, Lin D, Fan J (2017) Image privacy protection by identifying sensitive objects via deep multi-task learning. IEEE Trans Inf Forensics Secur 12:1005–1016CrossRef Yu J, Zhang B, Kuang Z, Lin D, Fan J (2017) Image privacy protection by identifying sensitive objects via deep multi-task learning. IEEE Trans Inf Forensics Secur 12:1005–1016CrossRef
54.
Zurück zum Zitat Yu J, Yang X, Gao F, Tao D (2016) Deep multimodal distance metric learning using click constraints for image ranking. IEEE Trans Cybern 12:1–11 Yu J, Yang X, Gao F, Tao D (2016) Deep multimodal distance metric learning using click constraints for image ranking. IEEE Trans Cybern 12:1–11
55.
Zurück zum Zitat Yu J, Rui Y, Tao D (2014) Click prediction for web image reranking using multimodal sparse coding. IEEE Trans Image Process 23:2019–2032MathSciNetCrossRef Yu J, Rui Y, Tao D (2014) Click prediction for web image reranking using multimodal sparse coding. IEEE Trans Image Process 23:2019–2032MathSciNetCrossRef
56.
Zurück zum Zitat Sun S, Zhou J (2014) A review of adaptive feature extraction and classification methods for EEG-based brain-computer interfaces. In: Proceedings of the international joint conference on neural networks, pp 1746–1753 Sun S, Zhou J (2014) A review of adaptive feature extraction and classification methods for EEG-based brain-computer interfaces. In: Proceedings of the international joint conference on neural networks, pp 1746–1753
57.
Zurück zum Zitat Sun S, Luo C, Chen J (2016) A review of natural language processing techniques for opinion mining systems. Inf Fusion 36:10–25CrossRef Sun S, Luo C, Chen J (2016) A review of natural language processing techniques for opinion mining systems. Inf Fusion 36:10–25CrossRef
Metadaten
Titel
Domain Adaptation with Twin Support Vector Machines
verfasst von
Xijiong Xie
Shiliang Sun
Huahui Chen
Jiangbo Qian
Publikationsdatum
19.12.2017
Verlag
Springer US
Erschienen in
Neural Processing Letters / Ausgabe 2/2018
Print ISSN: 1370-4621
Elektronische ISSN: 1573-773X
DOI
https://doi.org/10.1007/s11063-017-9775-3

Weitere Artikel der Ausgabe 2/2018

Neural Processing Letters 2/2018 Zur Ausgabe

Neuer Inhalt