Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
Published in:
Contextual Word Embeddings Clustering Through Multiway Analysis: A Comparative Study Read chapter Read first chapter

2023 | OriginalPaper | Chapter

Diverse Paraphrasing with Insertion Models for Few-Shot Intent Detection

Authors : Raphaël Chevasson, Charlotte Laclau, Christophe Gravier

Published in: Advances in Intelligent Data Analysis XXI

Publisher: Springer Nature Switzerland

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

In contrast to classic autoregressive generation, insertion-based models can predict in a order-free way multiple tokens at a time, which make their generation uniquely controllable: it can be constrained to strictly include an ordered list of tokens. We propose to exploit this feature in a new diverse paraphrasing framework: first, we extract important tokens or keywords in the source sentence; second, we augment them; third, we generate new samples around them by using insertion models. We show that the generated paraphrases are competitive with state of the art autoregressive paraphrasers, not only in diversity but also in quality. We further investigate their potential to create new pseudo-labelled samples for data augmentation, using a meta-learning classification framework, and find equally competitive result. In addition to proving non-autoregressive (NAR) viability for paraphrasing, we contribute our open-source framework as a starting point for further research into controllable NAR generation.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now
previous chapter Towards Data Science Design Patterns
next chapter LEMON: Alternative Sampling for More Faithful Explanation Through Local Surrogate Models
Footnotes
1
The Huggingface library [27] mostly uses this approach.
 
2
The Fairseq library [19] also uses this approach.
 
3
segtok: github.
 
4
YAKE: github.
 
5
More precisely, keeping “![end]”, “?[end]”, and “.[end]” as keywords.
 
6
We tested the lesker from nltk (→wordnet synset), bablify (→bablenet synset), getalp/​disambiguate, and ewiser (→wordnet synset) wich vastly outperformed others.
 
8
 
9
Bart-uni: paraphrases.
 
Literature
1.
Bevilacqua, M., Navigli, R.: Breaking through the 80% glass ceiling: raising the state of the art in word sense disambiguation by incorporating knowledge graph information. In: Proceedings of ACL, pp. 2854–2864. ACL (2020)
2.
Cao, Y., Wan, X.: DivGAN: towards diverse paraphrase generation via diversified generative adversarial network. In: Findings of the Association for Computational Linguistics (EMNLP 2020), pp. 2411–2421. ACL, Online (2020)
3.
Damodaran, P.: Parrot: paraphrase generation for NLU (2021)
4.
Dopierre, T., Gravier, C., Logerais, W.: PROTAUGMENT: unsupervised diverse short-texts paraphrasing for intent detection meta-learning. In: Proceedings of ACL-IJCNLP, pp. 2454–2466 (2021)
5.
Geng, R., Li, B., Li, Y., Zhu, X., Jian, P., Sun, J.: Induction networks for few-shot text classification. In: Proceedings of EMNLP-IJCNLP, pp. 3904–3913 (2019)
6.
Goyal, T., Durrett, G.: Neural syntactic preordering for controlled paraphrase generation. In: Proceedings of ACL, pp. 238–252. ACL (2020)
7.
Gu, J., Wang, C., Zhao, J.: Levenshtein transformer. In: NeurIPS, vol. 32 (2019)
8.
Guo, Y., Liao, Y., Jiang, X., Zhang, Q., Zhang, Y., Liu, Q.: Zero-shot paraphrase generation with multilingual language models. arXiv preprint arXiv:​1911.​03597 (2019)
9.
Guo, Z., et al.: Automatically paraphrasing via sentence reconstruction and round-trip translation. In: Zhou, Z.H. (ed.) Proceedings of IJCAI, pp. 3815–3821 (2021)
10.
Ippolito, D., Kriz, R., Sedoc, J., Kustikova, M., Callison-Burch, C.: Comparison of diverse decoding methods from conditional language models. In: Proceedings of ACL, pp. 3752–3762 (2019)
11.
Jorge, A., et al.: Text2story workshop - narrative extraction from texts. In: SIGIR Forum, pp. 150–152 (2018)
12.
Karimi, A., Rossi, L., Prati, A.: AEDA: an easier data augmentation technique for text classification. In: Findings of ACL: EMNLP, pp. 2748–2754. ACL (2021)
13.
Lewis, M., et al.: BART: denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In: Proceedings of ACL (2020)
14.
Mallinson, J., Sennrich, R., Lapata, M.: Paraphrasing revisited with neural machine translation. In: Proceedings of EACL, pp. 881–893 (2017)
15.
Malmi, E., Krause, S., Rothe, S., Mirylenka, D., Severyn, A.: Encode, tag, realize: high-precision text editing. In: Proceedings EMNLP-IJCNLP, pp. 5054–5065 (2019)
16.
Mehri, S., Eric, M., Hakkani-Tür, D.: Dialoglue: a natural language understanding benchmark for task-oriented dialogue (2020)
17.
Merchant, A., Rahimtoroghi, E., Pavlick, E., Tenney, I.: What happens to BERT embeddings during fine-tuning? In: Proceedings of the BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP, pp. 33–44. ACL (2020)
18.
Niu, T., Yavuz, S., Zhou, Y., Keskar, N.S., Wang, H., Xiong, C.: Unsupervised paraphrasing with pretrained language models. In: Proceedings of EMNLP, pp. 5136–5150. ACL (2021)
19.
Ott, M., et al.: fairseq: a fast, extensible toolkit for sequence modeling. In: Proceedings of NAACL-HLT: Demonstrations (2019)
20.
Qian, L., Qiu, L., Zhang, W., Jiang, X., Yu, Y.: Exploring diverse expressions for paraphrase generation. In: Proceedings of EMNLP-IJCNLP, pp. 3173–3182 (2019)
21.
Raffel, C., et al.: Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res. 21(140), 1–67 (2020)MathSciNetMATH
22.
Sharma, L., Graesser, L., Nangia, N., Evci, U.: Natural language understanding with the Quora question pairs dataset (2019)
23.
Stern, M., Chan, W., Kiros, J., Uszkoreit, J.: Insertion transformer: flexible sequence generation via insertion operations. In: Chaudhuri, K., Salakhutdinov, R. (eds.) Proceedings of ICML, vol. 97, pp. 5976–5985. PMLR (2019)
24.
Vijayakumar, A., et al.: Diverse beam search for improved description of complex scenes. In: Proceedings of AAAI, vol. 32, issue 1 (2018)
25.
Wei, J., Zou, K.: EDA: easy data augmentation techniques for boosting performance on text classification tasks. In: Proceedings of EMNLP-IJCNLP, pp. 6382–6388 (2019)
26.
Wieting, J., Mallinson, J., Gimpel, K.: Learning paraphrastic sentence embeddings from back-translated bitext. In: Proceedings of EMNLP, pp. 274–285. ACL, Copenhagen, Denmark (2017)
27.
Wolf, T., et al.: Transformers: state-of-the-art natural language processing. In: Proceedings of EMNLP: System Demonstrations, pp. 38–45. ACL (2020)
28.
Yang, Y., Zhang, Y., Tar, C., Baldridge, J.: PAWS-X: a cross-lingual adversarial dataset for paraphrase identification. In: Proceedings of EMNLP-IJCNLP, pp. 3687–3692. ACL, Hong Kong, China (2019)
29.
Zhang, Y., Wang, G., Li, C., Gan, Z., Brockett, C., Dolan, B.: POINTER: constrained progressive text generation via insertion-based generative pre-training. In: Proceedings of EMNLP, pp. 8649–8670. ACL (2020)
30.
Zhang, Y., Baldridge, J., He, L.: PAWS: paraphrase adversaries from word scrambling. In: Proceedings of NAACL/HLT, pp. 1298–1308. ACL (2019)
31.
Zhao, S., Wang, H.: Paraphrases and applications. In: Paraphrases and Applications-Tutorial Notes (Coling 2010), pp. 1–87 (2010)
32.
Zhou, J., Bhat, S.: Paraphrase generation: a survey of the state of the art. In: Proceedings of EMNLP, pp. 5075–5086. ACL (2021)
Metadata
Title
Diverse Paraphrasing with Insertion Models for Few-Shot Intent Detection
Authors
Raphaël Chevasson
Charlotte Laclau
Christophe Gravier
Copyright Year
2023
Book
Advances in Intelligent Data Analysis XXI

Print ISBN: 978-3-031-30046-2

Electronic ISBN: 978-3-031-30047-9

Copyright Year: 2023

DOI
https://doi.org/10.1007/978-3-031-30047-9_6

Premium Partner

    Image Credits