Effective domain awareness and adaptation approach via mask substructure for multi-domain neural machine translation

Multi-domain adaptation of neural machine translation (NMT) aims to learn a unified seq2seq framework based on multi-domain data. Domain corpus data mixing is one of the most important ways for multi-domain NMT, which has been widely explored in many recent works. However, due to the limitation of data mixing strategy, it often suffers from catastrophic forgetting problem or domain shift problem. To this end, we propose a domain-aware NMT with mask substructure. The mask substructure is employed in both Transformer-encoder and Transformer-decoder to capture domain-specific representations for each domain, then a domain fusion strategy is adopted to obtain a multi-domain adaptive NMT model. Our domain fusion framework could share domain-invariant knowledge and maintain domain-specific knowledge. We conduct extensive experiments on multi-domain NMT dataset, and the experimental results show significant improvements over the state-of-the-art (SOTA) approaches by up to 1.1 BLEU points on 8 domains and up to 4.5 BLEU points on an unseen domain. Moreover, the in-depth analysis shows that our model can also effectively alleviate both catastrophic forgetting and domain shift problems.