nach oben

International Journal of Computer Vision

Erschienen in:

19.11.2023

Adapting Across Domains via Target-Oriented Transferable Semantic Augmentation Under Prototype Constraint

verfasst von: Mixue Xie, Shuang Li, Kaixiong Gong, Yulin Wang, Gao Huang

Erschienen in: International Journal of Computer Vision | Ausgabe 4/2024

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

The demand for reducing label annotation cost and adapting to new data distributions gives rise to the emergence of domain adaptation (DA). DA aims to learn a model that performs well on the unlabeled or scarcely labeled target domain by transferring the rich knowledge from a related and well-annotated source domain. Existing DA methods mainly resort to learning domain-invariant representations with a source-supervised classifier shared by two domains. However, such a shared classifier may bias towards source domain, limiting its generalization capability on target data. To alleviate this issue, we present a target-oriented transferable semantic augmentation (T\(^2\)SA) method, which enhances the generalization ability of the classifier by training it with a target-like augmented domain, constructed by semantically augmenting source data towards target at the feature level in an implicit manner. Specifically, to equip the augmented domain with target semantics, we delicately design a class-wise multivariate normal distribution based on the statistics estimated from features to sample the transformation directions for source data. Moreover, we achieve the augmentation implicitly by minimizing the upper bound of the expected Angular-softmax loss over the augmented domain, which is of high efficiency. Additionally, to further ensure that the augmented domain can imitate target domain nicely and discriminatively, the prototype constraint is enforced on augmented features class-wisely, which minimizes the expected distance between augmented features and corresponding target prototype (i.e., average representation) in Euclidean space. As a general technique, T\(^2\)SA can be easily plugged into various DA methods to further boost their performances. Extensive experiments under single-source DA, multi-source DA and domain generalization scenarios validate the efficacy of T\(^2\)SA.

Vorheriger Artikel PartCom: Part Composition Learning for 3D Open-Set Recognition

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Nur mit Berechtigung zugänglich

For simplicity, the softmax loss is defined as the combination of the softmax function and cross-entropy loss in this paper, following (Liu et al., 2017b).

Antoniou, A., Storkey, A. J., & Edwards, H. A. (2017). Data augmentation generative adversarial networks. arXiv:1711.04340

Balaji, Y., Sankaranarayanan, S., & Chellappa, R. (2018). Metareg: Towards domain generalization using meta-regularization. In Proceedings of advances in neural information processing systems (pp. 1006–1016).

Bendavid, S., Blitzer, J., Crammer, K., et al. (2010). A theory of learning from different domains. Machine Learning, 79(1–2), 151–175.MathSciNetCrossRef

Carlucci, F. M., D’Innocente, A., Bucci, S., Caputo, B., & Tommasi, T. (2019). Domain generalization by solving jigsaw puzzles. In Proceedings of IEEE conference on computer vision and pattern recognition (pp. 2229–2238).

Chandler, D. (1987). Introduction to modern statistical. Mechanics Oxford University Press.

Creswell, A., White, T., Dumoulin, V., Arulkumaran, K., Sengupta, B., & Bharath, A. A. (2014), Generative adversarial nets. In Proceedings of advances in neural information processing systems (pp. 2672–2680).

Cui, S., Wang, S., Zhuo, J., Li, L., Huang, Q., & Tian, Q. (2021) Fast batch nuclear-norm maximization and minimization for robust domain adaptation. arXiv:2107.06154

Cui, S., Wang, S., Zhuo, J., Su, C., Huang, Q., & Tian, Q. (2020). Gradually vanishing bridge for adversarial domain adaptation. In IEEE conference on computer vision and pattern recognition.

Deng, J., Dong, W., Socher, R., Li, L. J., Li, K., & Fei-Fei, L. (2009). Imagenet: A large-scale hierarchical image database. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 248–255).

Der Maaten, L. V., Chen, M., Tyree, S., & Weinberger, K. (2013). Learning with marginalized corrupted features. In Proceedings of international conference on machine learning (pp. 410–418).

D’Innocente, A., & Caputo, B. (2018). Domain generalization with domain-specific aggregation modules. In German Conference on Pattern Recognition (pp. 187–198).

Dou, Q., Coelho de Castro, D., Kamnitsas, K., & Glocker, B. (2019). Domain generalization via model-agnostic learning of semantic features. In Proceedings of advances in neural information processing systems (pp. 6447–6458).

Ganin, Y., & Lempitsky, V. S. (2015). Unsupervised domain adaptation by backpropagation. In Proceedings of international conference on machine learning (pp. 1180–1189).

Ganin, Y., Ustinova, E., Ajakan, H., Germain, P., Larochelle, H., Laviolette, F., & Lempitsky, V. (2016). Domain-adversarial training of neural networks. Journal of Machine Learning Research, 17, 59:1-59:35.MathSciNet

Gao, J., Hua, Y., Hu, G., Wang, C., & Robertson, N. M. (2020). Reducing distributional uncertainty by mutual information maximisation and transferable feature learning. In Proceedings of the IEEE conference on computer vision (pp. 587–605).

Grandvalet, Y., & Bengio, Y. (2005). Semi-supervised learning by entropy minimization. In Proceedings of advances in neural information processing systems (pp. 529–536).

Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., & Courville, A. C. (2017). Improved training of wasserstein gans. In Proceedings of proceedings of advances in neural information processing systems (pp. 5769–5779).

He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770–778).

Hoffman, J., Tzeng, E., Park, T., Zhu, J. Y., Isola, P., Saenko, K., & Darrell, T. (2018) Cycada: Cycle-consistent adversarial domain adaptation. In International conference on machine learning (pp. 1994–2003).

Huang, S. W., Lin, C. T., Chen, S. P., Wu, Y. Y., Hsu, P. H., & Lai, S. H. (2018). Auggan: Cross domain adaptation with gan-based data augmentation. In Proceedings of the IEEE European conference on computer vision (pp. 731–744).

Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017). Densely connected convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 4700–4708).

Huang, Z., Wang, H., Xing, E. P., & Huang, D. (2020). Self-challenging improves cross-domain generalization. In Proceedings of the IEEE conference on computer vision (pp. 124–140).

Hull, J. J. (1994). A database for handwritten text recognition research. IEEE Transactions on Pattern Analysis and Machine Intelligence, 16(5), 550–554.CrossRef

Jin, Y., Wang, X., Long, M., & Wang, J. (2020). Minimum class confusion for versatile domain adaptation. In Proceedings of the IEEE conference on computer vision (pp. 464–480).

LeCun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86, 2278–2324.CrossRef

Lee, C. Y., Batra, T., Baig, M. H., & Ulbricht, D. (2019). Sliced wasserstein discrepancy for unsupervised domain adaptation. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 10285–10295).

Li, S., Liu, C. H., Xie, B., Su, L., Ding, Z., & Huang, G. (2019). Joint adversarial domain adaptation. In Proceedings of ACM international conference on multimedia (pp. 729–737).

Li, Y., Murias, M., & Dawson, G. (2018b). Extracting relationships by multi-domain matching. In Proceedings of advances in neural information processing systems (pp. 6799–6810).

Li, H., Pan, S. J., Wang, S., & Kot, A. C. (2018a). Domain generalization with adversarial feature learning. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 5400–5409).

Li, Y., Tian, X., Gong, M., Liu, Y., Liu, T., Zhang, K., & Tao, D. (2018c). Deep domain generalization via conditional invariant adversarial networks. In Proceedings of the IEEE conference on computer vision (pp. 647–663).

Li, S., Xie, M., Gong, K., Liu, C. H., Wang, Y., & Li, W. (2021d). Transferable semantic augmentation for domain adaptation. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 11516–11525).

Li, D., Yang, Y., Song, Y. Z., & Hospedales, T. M. (2017). Deeper, broader and artier domain generalization. In Proceedings of the IEEE international conference on computer vision (pp. 5543–5551).

Li, Y., Yuan, L., Chen, Y., Wang, P., & Vasconcelos, N. (2021e). Dynamic transfer for multi-source domain adaptation. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 10998–11007).

Li, M., Zhai, Y. M., Luo, Y. W., Ge, P. F., & Ren, C. X. (2020). Enhanced transport distance for unsupervised domain adaptation. In Proceedings of the IEEE conference on computer vision and pattern recognition.

Li, J., Du, Z., Zhu, L., Ding, Z., Lu, K., & Shen, H. T. (2021). Divergence-agnostic unsupervised domain adaptation by adversarial attacks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25, 1–1.

Li, S., Liu, C. H., Lin, Q., Wen, Q., Su, L., Huang, G., & Ding, Z. (2021). Deep residual correction network for partial domain adaptation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(7), 2329–2344.CrossRef

Liu, M.Y., Breuel, T., & Kautz, J. (2017a). Unsupervised image-to-image translation networks. In Proceedings of advances in neural information processing systems (pp. 700–708).

Liu, H., Long, M., Wang, J., & Jordan, M. (2019). Transferable adversarial training: A general approach to adapting deep classifiers. In Proceedings of international conference on machine learning (pp. 4013–4022).

Liu, W., Wen, Y., Yu, Z., Li, M., Raj, B., & Song, L. (2017b). Sphereface: Deep hypersphere embedding for face recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 6738–6746).

Li, S., Xie, B., Lin, Q., Liu, C. H., Huang, G., & Wang, G. (2021). Generalized domain conditioned adaptation network. IEEE Transactions on Pattern Analysis and Machine Intelligence, 8, 1–1.

Long, M., Cao, Y., Wang, J., & Jordan, M. (2015). Learning transferable features with deep adaptation networks. In Proceedings of international conference on machine learning (pp. 97–105).

Long, M., Cao, Z., Wang, J., & Jordan, M. I. (2018). Conditional adversarial domain adaptation. In Proceedings of advances in neural information processing systems (pp. 1647–1657).

Long, M., Zhu, H., Wang, J., & Jordan, M. I. (2016). Unsupervised domain adaptation with residual transfer networks. In Proceedings of advances in neural information processing systems (pp. 136–144).

Long, M., Zhu, H., Wang, J., & Jordan, M. I. (2017). Deep transfer learning with joint adaptation networks. In International conference on machine learning (pp. 2208–2217).

Lvd, M., & Hinton, G. (2008). Visualizing data using t-sne. Journal of Machine Learning Research, 9, 2579–2605.

Muandet, K., Balduzzi, D., & Schölkopf, B. (2013). Domain generalization via invariant feature representation. In Proceedings of international conference on machine learning (pp. 10–18).

Netzer, Y., Wang, T., Coates, A., Bissacco, A., Wu, B., & Ng, A. Y. (2011). Reading digits in natural images with unsupervised feature learning. In Proceedings of advances in neural information processing systems.

Pan, Y., Yao, T., Li, Y., Wang, Y., Ngo, C. W., & Mei, T. (2019). Transferrable prototypical networks for unsupervised domain adaptation. In Proceedings of the IEEE conference on computer vision and pattern recognition.

Pan, S. J., & Yang, Q. (2010). A survey on transfer learning. IEEE Transactions on Knowledge and Data Engineering, 22(10), 1345–1359.CrossRef

Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., & Chintala, S. (2019). Pytorch: An imperative style, high-performance deep learning library. Proceedings of Advances in Neural Information Processing Systems, 32, 8024–8035.

Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., & Duchesnay, E. (2012). Scikit-learn: Machine learning in python. Journal of Machine Learning Research, 12, 2830–2852.

Peng, X., Bai, Q., Xia, X., Huang, Z., Saenko, K., & Wang, B. (2019). Moment matching for multi-source domain adaptation. In Proceedings of the IEEE conference on computer vision (pp. 1406–1415).

Peng, X., Usman, B., Kaushik, N., Hoffman, J., Wang, D., & Saenko, K. (2017). Visda: The visual domain adaptation challenge. arXiv:1710.06924

Pinheiro, P. O. (2018). Unsupervised domain adaptation with similarity learning. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 8004–8013).

Ratner, A. J., Ehrenberg, H., Hussain, Z., Dunnmon, J., & Re, C. (2017). Learning to compose domain-specific transformations for data augmentation. In Proceedings of advances in neural information processing systems (pp. 3236–3246).

Ren, S., He, K., Girshick, R. B., et al. (2017). Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(6), 1137–1149.CrossRef

Révész, P. (2014). The laws of large numbers (Vol. 4). Academic Press.

Saenko, K., Kulis, B., Fritz, M., & Darrell, T. (2010). Adapting visual category models to new domains. In Proceedings of the IEEE European conference on computer vision (pp. 213–226).

Saito, K., Kim, D., Sclaroff, S., Darrell, T., & Saenko, K. (2019). Semi-supervised domain adaptation via minimax entropy. In Proceedings of the IEEE conference on computer vision (pp. 8049–8057).

Saito, K., Watanabe, K., Ushiku, Y., & Harada, T. (2018). Maximum classifier discrepancy for unsupervised domain adaptation. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 3723–3732).

Scalbert, M., Vakalopoulou, M., & Couzinié-Devy, F. (2021). Multi-source domain adaptation via supervised contrastive learning and confident consistency regularization. arXiv:2106.16093

Sun, B., & Saenko, K. (2016). Deep CORAL: Correlation alignment for deep domain adaptation. In Proceedings of the IEEE European conference on computer vision (pp. 443–450).

Tzeng, E., Hoffman, J., Saenko, K., & Darrell, T. (2017). Adversarial discriminative domain adaptation. In Proceedings of the IEEE conference on computer vision and pattern recognition (p. 4).

Upchurch, P., Gardner, J., Pleiss, G., Pless, R., Snavely, N., Bala, K., & Weinberger, K. (2017). Deep feature interpolation for image content changes. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 6090–6099).

Venkateswara, H., Eusebio, J., Chakraborty, S., & Panchanathan, S. (2017). Deep hashing network for unsupervised domain adaptation. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp 5018–5027).

Volpi, R., Morerio, P., Savarese, S., & Murino, V. (2017). Adversarial feature augmentation for unsupervised domain adaptation. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 5495–5504).

Vu, T. H., Jain, H., Bucher, M., Cord, M., & Perez, P. (2019). ADVENT: Adversarial entropy minimization for domain adaptation in semantic segmentation. In Proceedings of the IEEE conference on computer vision (pp. 2517–2526).

Wang, Y., Pan, X., Song, S., Zhang, H., Huang, G., & Wu, C. (2019). Implicit semantic data augmentation for deep networks. In Proceedings of advances in neural information processing systems (pp. 12614–12623).

Wang, F., Xiang, X., Cheng, J., & Yuille, A. L. (2017). Normface: L\(_{2}\) hypersphere embedding for face verification. In Proceedings of ACM international conference on multimedia (pp. 1041–1049).

Wang, H., Xu, M., Ni, B., & Zhang, W. (2020a). Learning to combine: Knowledge aggregation for multi-source domain adaptation. In Proceedings of the IEEE European conference on computer vision (pp. 727–744).

Wang, S., Yu, L., Li, C., Fu, C. W., & Heng, P. A. (2020b). Learning from extrinsic and intrinsic supervisions for domain generalization. In Proceedings of the IEEE conference on computer vision (pp. 159–176).

Wen, J., Greiner, R., & Schuurmans, D. (2020). Domain aggregation networks for multi-source domain adaptation. In Proceedings of international conference on machine learning (pp. 10214–10224).

Wu, Y., Inkpen, D., & El-Roby, A. (2020b). Dual mixup regularized learning for adversarial domain adaptation. In Proceedings of IEEE European conference on computer vision (pp. 540–555).

Wu, X., Zhou, Q., Yang, Z., Zhao, C., & Latecki, L. J. (2020a). Entropy minimization versus diversity maximization for domain adaptation. arXiv:2002.01690

Xu, Y., Bo, D., Zhang, L., & Zhang, Q. (2019). Self-ensembling attention networks: Addressing domain shift for semantic segmentation. In Proceedings of the AAAI conference on artificial intelligence (pp. 5581–5588).

Xu, R., Chen, Z., Zuo, W., Yan, J., & Lin, L. (2018). Deep cocktail network: Multi-source unsupervised domain adaptation with category shift. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 3964–3973).

Xu, M., Zhang, J., Ni, B., Li, T., Wang, C., Tian, Q., & Zhang, W. (2020). Adversarial domain adaptation with domain mixup. In Proceedings of the AAAI conference on artificial intelligence (pp. 6502–6509).

Xu, Q., Zhang, R., Zhang, Y., Wang, Y., & Tian, Q. (2021). A Fourier-based framework for domain generalization. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 14383–14392).

Yang, L., Balaji, Y., Lim, S., & Shrivastava, A. (2020). Curriculum manager for source selection in multi-source domain adaptation. In Proceedings of IEEE European conference on computer vision (pp. 608–624).

You, K., Wang, X., Long, M., & Jordan, M. (2019). Towards accurate model selection in deep unsupervised domain adaptation. In Proceedings of international conference on machine learning (pp. 7124–7133).

Zhang, Y., Liu, T., Long, M., & Jordan, M. (2019a). Bridging theory and algorithm for domain adaptation. In Proceedings of international conference on machine learning (pp. 7404–7413).

Zhang, Y., Tang, H., Jia, K., & Tan, M. (2019b). Domain-symmetric networks for adversarial domain adaptation. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 5031–5040).

Zhang, W., Xu, D., Ouyang, W., & Li, W. (2021). Self-paced collaborative and adversarial network for unsupervised domain adaptation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(6), 2047–2061.CrossRef

Zhao, S., Gong, M., Liu, T., Fu, H., & Tao, D. (2020). Domain generalization via entropy regularization. In Advances in neural information processing systems.

Zhao, H., Zhang, S., Wu, G., Moura, J. M., Costeira, J. P., & Gordon, G. J. (2018). Adversarial multiple source domain adaptation. In Proceedings of advances in neural information processing systems (pp. 8568–8579).

Zhou, K., Yang, Y., Hospedales, T. M., & Xiang, T. (2020). Learning to generate novel domains for domain generalization. In Proceedings of IEEE European conference on computer vision (pp 561–578).

Zhu, Y., Zhuang, F., & Wang, D. (2019). Aligning domain-specific distribution and classifier for cross-domain classification from multiple sources. In AAAI conference on artificial intelligence (pp. 5989–5996).

Titel: Adapting Across Domains via Target-Oriented Transferable Semantic Augmentation Under Prototype Constraint
verfasst von: Mixue Xie
Shuang Li
Kaixiong Gong
Yulin Wang
Gao Huang
Publikationsdatum: 19.11.2023
Verlag: Springer US
Erschienen in: International Journal of Computer Vision / Ausgabe 4/2024
Print ISSN: 0920-5691
Elektronische ISSN: 1573-1405
DOI: https://doi.org/10.1007/s11263-023-01944-1

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Wirtschaft"

Springer Professional "Technik"

Weitere Artikel der Ausgabe 4/2024

Harmonizing Base and Novel Classes: A Class-Contrastive Approach for Generalized Few-Shot Segmentation

Learning Robust Multi-scale Representation for Neural Radiance Fields from Unposed Images

Language-Aware Soft Prompting: Text-to-Text Optimization for Few- and Zero-Shot Adaptation of V &L Models

Guest Editorial: Special Issue on the Promises and Dangers of Large Vision Models

SegViT v2: Exploring Efficient and Continual Semantic Segmentation with Plain Vision Transformers

DIVOTrack: A Novel Dataset and Baseline Method for Cross-View Multi-Object Tracking in DIVerse Open Scenes

Premium Partner