Skip to main content
Fachgebiete
Automobil + Motoren Bauwesen + Immobilien Business IT + Informatik Elektrotechnik + Elektronik Energie + Nachhaltigkeit Finance + Banking Management + Führung Marketing + Vertrieb Maschinenbau + Werkstoffe Versicherung + Risiko
DE
EN
Themenseiten
Organisationspsychologie Projektmanagement Marketing Smart Manufacturing
Bücher
Zeitschriften
Weitere Formate
Podcasts Webinare Technik Webinare Wirtschaft Kongresse Veranstaltungskalender Awards
Jetzt Einzelzugang starten
Zugang für Unternehmen
Referenzkunden
MTZ-Fachkongress

Antriebe und Energiesysteme von morgen


Austausch von Automobil- und Energiewirtschaft

Am 14./15. Mai geht es in Chemnitz um die Mobilitätswende durch Elektro- und Wasserstofffahrzeuge.
Erweiterte Suche
Anmelden
nach oben
Erschienen in:
Contextual Word Embeddings Clustering Through Multiway Analysis: A Comparative Study Kapitel lesen Erstes Kapitel lesen

2023 | OriginalPaper | Buchkapitel

Diverse Paraphrasing with Insertion Models for Few-Shot Intent Detection

verfasst von : Raphaël Chevasson, Charlotte Laclau, Christophe Gravier

Erschienen in: Advances in Intelligent Data Analysis XXI

Verlag: Springer Nature Switzerland

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

In contrast to classic autoregressive generation, insertion-based models can predict in a order-free way multiple tokens at a time, which make their generation uniquely controllable: it can be constrained to strictly include an ordered list of tokens. We propose to exploit this feature in a new diverse paraphrasing framework: first, we extract important tokens or keywords in the source sentence; second, we augment them; third, we generate new samples around them by using insertion models. We show that the generated paraphrases are competitive with state of the art autoregressive paraphrasers, not only in diversity but also in quality. We further investigate their potential to create new pseudo-labelled samples for data augmentation, using a meta-learning classification framework, and find equally competitive result. In addition to proving non-autoregressive (NAR) viability for paraphrasing, we contribute our open-source framework as a starting point for further research into controllable NAR generation.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Jetzt informieren
Vorheriges Kapitel Towards Data Science Design Patterns
Nächstes Kapitel LEMON: Alternative Sampling for More Faithful Explanation Through Local Surrogate Models
Fußnoten
1
The Huggingface library [27] mostly uses this approach.
 
2
The Fairseq library [19] also uses this approach.
 
3
segtok: github.
 
4
YAKE: github.
 
5
More precisely, keeping “![end]”, “?[end]”, and “.[end]” as keywords.
 
6
We tested the lesker from nltk (→wordnet synset), bablify (→bablenet synset), getalp/​disambiguate, and ewiser (→wordnet synset) wich vastly outperformed others.
 
8
 
9
Bart-uni: paraphrases.
 
Literatur
1.
Bevilacqua, M., Navigli, R.: Breaking through the 80% glass ceiling: raising the state of the art in word sense disambiguation by incorporating knowledge graph information. In: Proceedings of ACL, pp. 2854–2864. ACL (2020)
2.
Cao, Y., Wan, X.: DivGAN: towards diverse paraphrase generation via diversified generative adversarial network. In: Findings of the Association for Computational Linguistics (EMNLP 2020), pp. 2411–2421. ACL, Online (2020)
3.
Damodaran, P.: Parrot: paraphrase generation for NLU (2021)
4.
Dopierre, T., Gravier, C., Logerais, W.: PROTAUGMENT: unsupervised diverse short-texts paraphrasing for intent detection meta-learning. In: Proceedings of ACL-IJCNLP, pp. 2454–2466 (2021)
5.
Geng, R., Li, B., Li, Y., Zhu, X., Jian, P., Sun, J.: Induction networks for few-shot text classification. In: Proceedings of EMNLP-IJCNLP, pp. 3904–3913 (2019)
6.
Goyal, T., Durrett, G.: Neural syntactic preordering for controlled paraphrase generation. In: Proceedings of ACL, pp. 238–252. ACL (2020)
7.
Gu, J., Wang, C., Zhao, J.: Levenshtein transformer. In: NeurIPS, vol. 32 (2019)
8.
Guo, Y., Liao, Y., Jiang, X., Zhang, Q., Zhang, Y., Liu, Q.: Zero-shot paraphrase generation with multilingual language models. arXiv preprint arXiv:​1911.​03597 (2019)
9.
Guo, Z., et al.: Automatically paraphrasing via sentence reconstruction and round-trip translation. In: Zhou, Z.H. (ed.) Proceedings of IJCAI, pp. 3815–3821 (2021)
10.
Ippolito, D., Kriz, R., Sedoc, J., Kustikova, M., Callison-Burch, C.: Comparison of diverse decoding methods from conditional language models. In: Proceedings of ACL, pp. 3752–3762 (2019)
11.
Jorge, A., et al.: Text2story workshop - narrative extraction from texts. In: SIGIR Forum, pp. 150–152 (2018)
12.
Karimi, A., Rossi, L., Prati, A.: AEDA: an easier data augmentation technique for text classification. In: Findings of ACL: EMNLP, pp. 2748–2754. ACL (2021)
13.
Lewis, M., et al.: BART: denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In: Proceedings of ACL (2020)
14.
Mallinson, J., Sennrich, R., Lapata, M.: Paraphrasing revisited with neural machine translation. In: Proceedings of EACL, pp. 881–893 (2017)
15.
Malmi, E., Krause, S., Rothe, S., Mirylenka, D., Severyn, A.: Encode, tag, realize: high-precision text editing. In: Proceedings EMNLP-IJCNLP, pp. 5054–5065 (2019)
16.
Mehri, S., Eric, M., Hakkani-Tür, D.: Dialoglue: a natural language understanding benchmark for task-oriented dialogue (2020)
17.
Merchant, A., Rahimtoroghi, E., Pavlick, E., Tenney, I.: What happens to BERT embeddings during fine-tuning? In: Proceedings of the BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP, pp. 33–44. ACL (2020)
18.
Niu, T., Yavuz, S., Zhou, Y., Keskar, N.S., Wang, H., Xiong, C.: Unsupervised paraphrasing with pretrained language models. In: Proceedings of EMNLP, pp. 5136–5150. ACL (2021)
19.
Ott, M., et al.: fairseq: a fast, extensible toolkit for sequence modeling. In: Proceedings of NAACL-HLT: Demonstrations (2019)
20.
Qian, L., Qiu, L., Zhang, W., Jiang, X., Yu, Y.: Exploring diverse expressions for paraphrase generation. In: Proceedings of EMNLP-IJCNLP, pp. 3173–3182 (2019)
21.
Raffel, C., et al.: Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res. 21(140), 1–67 (2020)MathSciNetMATH
22.
Sharma, L., Graesser, L., Nangia, N., Evci, U.: Natural language understanding with the Quora question pairs dataset (2019)
23.
Stern, M., Chan, W., Kiros, J., Uszkoreit, J.: Insertion transformer: flexible sequence generation via insertion operations. In: Chaudhuri, K., Salakhutdinov, R. (eds.) Proceedings of ICML, vol. 97, pp. 5976–5985. PMLR (2019)
24.
Vijayakumar, A., et al.: Diverse beam search for improved description of complex scenes. In: Proceedings of AAAI, vol. 32, issue 1 (2018)
25.
Wei, J., Zou, K.: EDA: easy data augmentation techniques for boosting performance on text classification tasks. In: Proceedings of EMNLP-IJCNLP, pp. 6382–6388 (2019)
26.
Wieting, J., Mallinson, J., Gimpel, K.: Learning paraphrastic sentence embeddings from back-translated bitext. In: Proceedings of EMNLP, pp. 274–285. ACL, Copenhagen, Denmark (2017)
27.
Wolf, T., et al.: Transformers: state-of-the-art natural language processing. In: Proceedings of EMNLP: System Demonstrations, pp. 38–45. ACL (2020)
28.
Yang, Y., Zhang, Y., Tar, C., Baldridge, J.: PAWS-X: a cross-lingual adversarial dataset for paraphrase identification. In: Proceedings of EMNLP-IJCNLP, pp. 3687–3692. ACL, Hong Kong, China (2019)
29.
Zhang, Y., Wang, G., Li, C., Gan, Z., Brockett, C., Dolan, B.: POINTER: constrained progressive text generation via insertion-based generative pre-training. In: Proceedings of EMNLP, pp. 8649–8670. ACL (2020)
30.
Zhang, Y., Baldridge, J., He, L.: PAWS: paraphrase adversaries from word scrambling. In: Proceedings of NAACL/HLT, pp. 1298–1308. ACL (2019)
31.
Zhao, S., Wang, H.: Paraphrases and applications. In: Paraphrases and Applications-Tutorial Notes (Coling 2010), pp. 1–87 (2010)
32.
Zhou, J., Bhat, S.: Paraphrase generation: a survey of the state of the art. In: Proceedings of EMNLP, pp. 5075–5086. ACL (2021)
Metadaten
Titel
Diverse Paraphrasing with Insertion Models for Few-Shot Intent Detection
verfasst von
Raphaël Chevasson
Charlotte Laclau
Christophe Gravier
Copyright-Jahr
2023
Buch
Advances in Intelligent Data Analysis XXI

Print ISBN: 978-3-031-30046-2

Electronic ISBN: 978-3-031-30047-9

Copyright-Jahr: 2023

DOI
https://doi.org/10.1007/978-3-031-30047-9_6

Premium Partner

    Bildnachweise