Skip to main content
SpringerLink
Log in

Joint discriminative subspace and distribution adaptation for unsupervised domain adaptation

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

In traditional machine learning algorithms, the classification models are learned on the training data (source domain) to reuse for labelling the test data (target domain) where the training and test samples are from the same distributions. However in nowadays applications, the existence of distribution shift across the source and target doamins degrades the model performance, significantly. Domain adaptation methods have been proposed to compensate domain shift problem by aligning the distributions across the source and target domains under various adaptation strategies. This paper addresses the robust image classification problem for unsupervised domain adaptation. Specifically, following three methods are proposed: Discriminative Subspace Learning (DSL), Joint Geometrical and Statistical Distribution Adaptation (GSDA), and Joint Subspace and Distribution Adaptation (DSL-GSDA). DSL is a subspace centric method that aligns the specific and shared features across domains. Indeed, DSL finds two projections to map the source and target data into independent subspaces by aligning the discriminant and global structures of domains. GSDA trends to find an adaptive classifier through statistical and geometrical distribution alignment and minimizes the prediction error. DSL-GSDA, as a combination of DSL and GSDA, consists of two subspace and distribution adaptation levels. DSL-GSDA uses DSL to build two aligned subspaces of source and target domains. The distributions of source and target data in new subspaces is adapted via GSDA. The proposed methods are evaluated on benchmark visual datasets for object, digit and face recongnition tasks. Visual datasets consist of image domains that have been captured under various real-world conditions where the domain shift is unavoidable. The experiment results show that DSL, GSDA and DSL-GSDA outperform other state-of-the-art domain adaptation methods by 6.19%, 1.48% and 1.99% improvement, respectively. Our source code is available at https://github.com/jtahmores/DSLGSDA (https://github.com/jtahmores/DSLGSDA).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Shi Y, Sha F (2012) Information-theoretical learning of discriminative clusters for unsupervised domain adaptation. arXiv:1206.6438

  2. Gong B, Grauman K, Sha F (2013) Reshaping visual datasets for domain adaptation. In: Advances in Neural Information Processing Systems, pp 1286–1294

  3. Jhuo IH, Liu D, Lee DT, Chang SF (2012) Robust visual domain adaptation with low-rank reconstruction. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, pp 2168–2175

  4. Daume H III, Marcu D (2006) Domain adaptation for statistical classifiers. J Artif Intell Res 26:101–126

    Article  MathSciNet  Google Scholar 

  5. Patel VM, Gopalan R, Li R, Chellappa R (2015) Visual domain adaptation: a survey of recent advances. IEEE Signal Process Mag 32(3):53–69

    Article  Google Scholar 

  6. Hana D, Liu Q, Fan W (2017) A New Image Classification Method Using CNN transfer learning and Web Data Augmentation. Expert Systems with Applications

  7. Luo Z, Hu J, Deng W, Shen H (2018) Deep unsupervised domain adaptation for face recognition. In: 2018 13th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2018). IEEE, pp 453–457

  8. Wen G, Chen H, Cai D, He X (2018) Improving face recognition with domain adaptation. Neurocomputing 287:45–51

    Article  Google Scholar 

  9. Blitzer J, Dredze M, Pereira F (2007) Biographies, bollywood, boom-boxes and blenders: Domain adaptation for sentiment classification. In: ACL, vol 7, pp 440–447

  10. Glorot X, Bordes A, Bengio Y (2011) Domain adaptation for large-scale sentiment classification: a deep learning approach. In: Proceedings of the 28th international conference on machine learning (ICML-11), pp 513–520

  11. Sun Y, Tzeng E, Darrell T, Efros AA (2019) Unsupervised Domain Adaptation through Self-Supervision. arXiv:1909.11825

  12. Ciga O, Chen J, Martel A (2019) Multi-layer Domain Adaptation for Deep Convolutional Networks. In: Domain Adaptation and Representation Transfer and Medical Image Learning with Less Labels and Imperfect Data. Springer, Cham, pp 20–27

  13. Yang J, Yan R, Hauptmann AG (2007) Cross-domain video concept detection using adaptive svms. In: Proceedings of the 15th ACM international conference on Multimedia. ACM, pp 188–197

  14. Duan L, Xu D, Tsang IWH, Luo J (2012) Visual event recognition in videos by learning from web data. IEEE Trans Pattern Anal Mach Intell 34(9):1667–1680

    Article  Google Scholar 

  15. Saenko K, Kulis B, Fritz M, Darrell T (2010) Adapting visual category models to new domains. In: European conference on computer vision. Springer, Berlin, pp 213–226

  16. Donahue J, Jia Y, Vinyals O, Hoffman J, Zhang N, Tzeng E, Darrell T (2014) Decaf: a deep convolutional activation feature for generic visual recognition. In: International conference on machine learning, pp 647–655

  17. LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc. IEEE 86(11):2278–2324

    Article  Google Scholar 

  18. Griffin G, Holub A, Perona P (2007) Caltech-256 object category dataset

  19. Sim T, Baker S, Bsat M (2001) The CMU pose, illumination and expression (PIE) database of human faces. Carnegie Mellon University, The Robotics Institute

  20. Blitzer J, McDonald R, Pereira F (2006) July) Domain adaptation with structural correspondence learning. In: Proceedings of the 2006 conference on empirical methods in natural language processing. Association for Computational Linguistics, pp 120–128

  21. Jiang J, Zhai C (2007) Instance weighting for domain adaptation in NLP. In: Proceedings of the 45th annual meeting of the association of computational linguistics, pp 264–271

  22. Bruzzone L, Marconcini M (2010) Domain adaptation problems: a DASVM classification technique and a circular validation strategy. IEEE Trans Pattern Anal Mach Intell 32(5):770–87

    Article  Google Scholar 

  23. Sun Q, Chattopadhyay R, Panchanathan S, Ye J (2011) A two-stage weighting framework for multi-source domain adaptation. In: Advances in neural information processing systems, pp 505–513

  24. Long M, Wang J, Ding G, Pan SJ, Philip SY (2014) Adaptation regularization: a general framework for transfer learning. IEEE Trans Knowl Data Eng 26(5):1076–89

    Article  Google Scholar 

  25. Luo L, Chen L, Hu S, Lu Y, Wang X (2017) Discriminative and geometry aware unsupervised domain adaptation. arXiv:1712.10042

  26. Pan SJ, Tsang IW, Kwok JT, Yang Q (2011) Domain adaptation via transfer component analysis. IEEE Trans Neural Netw 22(2):199–210

    Article  Google Scholar 

  27. Long M, Wang J, Ding G, Sun J, Philip SY (2013, December) Transfer feature learning with joint distribution adaptation. In: 2013 IEEE International Conference on Computer Vision (ICCV). IEEE, pp 2200–2207

  28. Tahmoresnezhad J, Hashemi S (2017) Visual domain adaptation via transfer feature learning. Knowl Inf Syst 50(2):585–605

    Article  Google Scholar 

  29. Xu Y, Fang X, Wu J, Li X, Zhang D (2016) Discriminative transfer subspace learning via low-rank and sparse representation. IEEE Trans Image Process 25(2):850–63

    Article  MathSciNet  Google Scholar 

  30. Luo L, Wang X, Hu S, Chen L (2017) Robust data geometric structure aligned close yet discriminative domain adaptation. arXiv:1705.08620

  31. Wang J, Chen Y, Hao S, Feng W, Shen Z (2017) Balanced distribution adaptation for transfer learning. In: 2017 IEEE International Conference on Data Mining (ICDM). IEEE, pp 1129–1134

  32. Ghifary M, Balduzzi D, Kleijn WB, Zhang M (2017) Scatter component analysis: a unified framework for domain adaptation and domain generalization. IEEE Trans Pattern Anal Mach Intell 39(7):1414–30

    Article  Google Scholar 

  33. Liu J, Li J, Lu K (2018) Coupled local–global adaptation for multi-source transfer learning. Neurocomputing 275:247–54

    Article  Google Scholar 

  34. Liang J, He R, Sun Z, Tan T (2018) Aggregating randomized clustering-promoting invariant projections for domain adaptation. IEEE Trans Pattern Anal Mach Intell 41(5):1027–1042

    Article  Google Scholar 

  35. Fernando B, Habrard A, Sebban M (2013) Tuytelaars T (2013, December) Unsupervised visual domain adaptation using subspace alignment. In: Computer Vision (ICCV) IEEE International Conference. IEEE, pp 2960–2967

  36. Sun B, Saenko K (2015) Subspace distribution alignment for unsupervised domain adaptation. In: BMVC, pp 24–1

  37. Zhang J, Li W, Ogunbona P (2017) Joint geometrical and statistical alignment for visual domain adaptation. arXiv:1705.05498

  38. Gholenji E, Tahmoresnezhad J (2019) Joint local and statistical discriminant Llearning via feature alignment. Signal, Image and Video Processing. https://doi.org/10.1007/s11760-019-01587-1

  39. Rezaei S, Tahmoresnezhad J (2019) Discriminative and domain invariant subspace alignment for visual tasks. Iran Journal of Computer Science 2:219–230. https://doi.org/10.1007/s42044-019-00037-y

    Article  Google Scholar 

  40. Mardani M, Tahmoresnezhad J (2018) Joint Distribution Adaptation via Feature and Model Matching. Scientia Iranica

  41. Tahmoresnezhad J, Hashemi S (2015) A generalized kernel-based random k-samplesets method for transfer learning. Iran J Sci Technol Trans Electr Eng 39:193–207

    Google Scholar 

  42. Abdi H, Williams LJ (2010) Principal component analysis. Wiley Interdiscip Rev Comput Stat 2(4):433–59

    Article  Google Scholar 

  43. Wang J, Feng W, Chen Y, Yu H, Huang M, Yu PS (2018) Visual domain adaptation with manifold embedded distribution alignment. In: 2018 ACM Multimedia Conference on Multimedia Conference. ACM, pp 402–410

  44. Gopalan R, Li R, Chellappa R (2011) Domain adaptation for object recognition: An unsupervised approach. In: 2011 international conference on computer vision. IEEE, pp 999–1006

  45. Welling M (2005) Fisher linear discriminant analysis. Department of Computer Science, University of Toronto 3(1)

  46. McLachlan GJ (2004) Discriminant analysis and statistical pattern recognition, John Wiley and Sons, 544

  47. Harel M, Mannor S (2010) Learning from multiple outlooks. arXiv:1005.0027

  48. Alvarez MA, Rosasco L, Lawrence ND (2012) Kernels for vector-valued functions: a review. Found Trends Mach Learn 4(3):195–266

    Article  Google Scholar 

  49. Schölkopf B, Herbrich R, Smola AJ (2001) A generalized representer theorem. In: International conference on computational learning theory, pp 416–426

  50. Gretton A, Borgwardt K, Rasch MJ, Scholkopf B, Smola AJ (2008) A kernel method for the two-sample problem. arXiv:0805.2368

  51. Belkin M, Niyogi P, Sindhwani V (2006) Manifold regularization: a geometric framework for learning from labeled and unlabeled examples. J Mach Learni Res 7(Nov):2399–434

    MathSciNet  MATH  Google Scholar 

  52. Li J, Lu K, Huang Z, Zhu L, Shen HT (2018) Transfer independently together: a generalized framework for domain adaptation. IEEE Trans Cybern 49(6):2144–2155

    Article  Google Scholar 

  53. Li S, Song S, Huang G, Ding Z, Wu C (2018) Domain invariant and class discriminative feature learning for visual domain adaptation. IEEE Trans Image Process 27(9):4260–4273

    Article  MathSciNet  Google Scholar 

  54. Uzair M, Mian A (2016) Blind domain adaptation with augmented extreme learning machine features. IEEE Trans Cybern 47(3):651–660

    Article  Google Scholar 

  55. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105

  56. Long M, Cao Y, Wang J, Jordan MI (2015) Learning transferable features with deep adaptation networks. arXiv:1502.02791

  57. Tzeng E, Hoffman J, Zhang N, Saenko K, Darrell T (2014) Deep domain confusion: Maximizing for domain invariance. arXiv:1412.3474

  58. Lu H, Zhang L, Cao Z, Wei W, Xian K, Shen C, van den Hengel A (2017) When unsupervised domain adaptation meets tensor representations. In: Proceedings of the IEEE International Conference on Computer Vision, pp 599–608

  59. Gholami B, Pavlovic V (2017) Punda: Probabilistic unsupervised domain adaptation for knowledge transfer across visual categories. In: Proceedings of the IEEE International Conference on Computer Vision, pp 3581–3590

Download references

Author information

Authors and Affiliations

  1. Faculty of IT and Computer Engineering, Urmia University of Technology, Urmia, Iran

    Elahe Gholenji & Jafar Tahmoresnezhad

Authors
  1. Elahe Gholenji
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jafar Tahmoresnezhad
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jafar Tahmoresnezhad.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gholenji, E., Tahmoresnezhad, J. Joint discriminative subspace and distribution adaptation for unsupervised domain adaptation. Appl Intell 50, 2050–2066 (2020). https://doi.org/10.1007/s10489-019-01610-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-019-01610-5

Keywords

Search

Navigation