Top

Knowledge and Information Systems

12-02-2025 | Regular Paper

RDPNet: a multi-stage summary generation network for long chat dialogues

Authors: Shihao Yang, Shaoru Zhang, Huayu Zhang, Chih-Cheng Hung, Fengqin Yang, Shuhua Liu

Published in: Knowledge and Information Systems

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

With the rapid development of human–computer interaction and natural language generation technology, chat dialogue summarization has attracted extensive attention from researchers, which aims to obtain significant information from chat history. The current chat dialogue summary models mainly focus on short dialogues due to the limitation of input text length. This paper proposes a RDPNet model, an effective multi-stage network for long chat dialogues. The method first uses a Retriever to select significant sentences from the long input dialogue, which not only can shorten the length of the source text, but also improve the structure of the chat. Secondly, a DialoGPT annotator is adopted to label the extracted important sentences so that it can further improve the structure of the dialogue. Finally, a large-scale pre-trained generation model ProphetNet is adopted to generate a concise dialogue summary. The experimental results demonstrate that RDPNet outperforms the state-of-the-art methods on three long chat summarization datasets DIALSUMM, SAMSum and TWEETSUMM and verify the effectiveness for long chat summarization.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

https://huggingface.co/microsoft/prophetnet-large-uncased.

https://huggingface.co/models?sort=trending&search=llama+3.2.

https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.3.

Gurevych I, Strube M (2004) Semantic similarity applied to spoken dialogue summarization. In: COLING 2004: proceedings of the 20th international conference on computational linguistics, pp. 764–770

Shang G, Ding W, Zhang Z, Tixier A, Meladianos P, Vazirgiannis M, Lorré J-P (2018) Unsupervised abstractive meeting summarization with multi-sentence compression and budgeted submodular maximization. In: Proceedings of the 56th annual meeting of the association for computational linguistics (Volume 1: Long Papers), pp. 664–674

Liu C, Wang P, Xu J, Li Z, Ye J (2019) Automatic dialogue summary generation for customer service. In: Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pp. 1957–1965

Gliwa B, Mochol I, Biesek M, Wawer A (2019) Samsum corpus: a human-annotated dialogue dataset for abstractive summarization. In: Proceedings of the 2nd workshop on new frontiers in summarization, pp. 70–79

Goo C-W, Chen Y-N (2018) Abstractive dialogue summarization with sentence-gated modeling optimized by dialogue acts. In: 2018 IEEE Spoken language technology workshop (SLT), pp. 735–742 . IEEE

Chen J, Yang D (2020) Multi-view sequence-to-sequence models with conversational structure for abstractive dialogue summarization. In: Proceedings of the 2020 conference on empirical methods in natural language processing (EMNLP), pp. 4106–4118

Wu C-S, Liu L, Liu W, Stenetorp P, Xiong C (2021) Controllable abstractive dialogue summarization with sketch supervision. Find Assoc Comput Linguist: ACL-IJCNLP 2021:5108–5122MATH

Lei Y, Yan Y, Zeng Z, He K, Zhang X, Xu W (2021) Hierarchical speaker-aware sequence-to-sequence model for dialogue summarization. In: ICASSP 2021-2021 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp. 7823–7827 . IEEE

Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser L, Polosukhin I (2017) Attention is all you need.(nips), 2017. arXiv preprint arXiv:1706.03762 10, 0140525–16001837

10.

Beltagy I, Peters ME, Cohan A (2020) Longformer: The long-document transformer. arXiv preprint arXiv:2004.05150

11.

Zaheer M, Guruganesh G, Dubey A, Ainslie J, Alberti C, Ontanon S, Pham P, Ravula A, Wang Q, Yang L, et al.: (2020) Big bird: transformers for longer sequences. In: Proceedings of the 34th international conference on neural information processing systems, pp. 17283–17297

12.

Zhang Y, Ni A, Yu T, Zhang R, Zhu C, Deb B, Celikyilmaz A, Awadallah AH, Radev D (2021) An exploratory study on long dialogue summarization: What works and what’s next. In: 2021 Findings of the association for computational linguistics, findings of ACL: EMNLP 2021, pp. 4426–4433 . Association for Computational Linguistics (ACL)

13.

Rohde T, Wu X, Liu Y (2021) Hierarchical learning for generation with long source sequences. arXiv preprint arXiv:2104.07545

14.

Feng X, Feng X, Qin L, Qin B, Liu T (2021) Language model as an annotator: exploring dialogpt for dialogue summarization. In: Proceedings of the 59th annual meeting of the association for computational linguistics and the 11th international joint conference on natural language processing (Volume 1: Long Papers), pp. 1479–1491

15.

Qi W, Yan Y, Gong Y, Liu D, Duan N, Chen J, Zhang R, Zhou M (2020) Prophetnet: predicting future n-gram for sequence-to-sequencepre-training. Find Assoc Comput Linguist: EMNLP 2020:2401–2410MATH

16.

Lin C-Y (2004) ROUGE: A package for automatic evaluation of summaries. In: Text summarization branches out, pp. 74–81. Association for computational linguistics, Barcelona, Spain . https://aclanthology.org/W04-1013

17.

Chen Y, Liu Y, Chen L, Zhang Y (2021) Dialogsum: a real-life scenario dialogue summarization dataset. Find Assoc Comput Linguist: ACL-IJCNLP 2021:5062–5074MATH

18.

Feigenblat G, Gunasekara C, Sznajder B, Joshi S, Konopnicki D, Aharonov R (2021) TWEETSUMM - a dialog summarization dataset for customer service. In: Moens, M.-F., Huang, X., Specia, L., Yih, S.W.-t. (eds.) Findings of the association for computational linguistics: EMNLP 2021, pp. 245–260. Association for computational linguistics, Punta Cana, Dominican Republic . https://doi.org/10.18653/v1/2021.findings-emnlp.24 . https://aclanthology.org/2021.findings-emnlp.24

19.

Paice CD (1990) Constructing literature abstracts by computer: techniques and prospects. Inform Process Manag 26(1):171–186CrossRefMATH

20.

See A, Liu PJ, Manning CD (2017) Get to the point: Summarization with pointer-generator networks. In: Proceedings of the 55th annual meeting of the association for computational linguistics (Volume 1: Long Papers), pp. 1073–1083

21.

Cohan A, Dernoncourt F, Kim DS, Bui T, Kim S, Chang W, Goharian N (2018) A discourse-aware attention model for abstractive summarization of long documents. In: Proceedings of the 2018 conference of the north american chapter of the association for computational linguistics: human language technologies, Volume 2 (Short Papers), pp. 615–621

22.

Riedhammer K, Favre B, Hakkani-Tur D (2008) A keyphrase based approach to interactive meeting summarization. In: 2008 IEEE Spoken language technology workshop, pp. 153–156 . IEEE

23.

Liu F, Liu Y (2010) Exploring speaker characteristics for meeting summarization. In: INTERSPEECH, pp. 2518–2521

24.

Zhu C, Xu R, Zeng M, Huang X (2020) A hierarchical network for abstractive meeting summarization with cross-domain pretraining. Find Assoc Comput Linguist: EMNLP 2020:194–203MATH

25.

26.

Zhong M, Liu Y, Xu Y, Zhu C, Zeng M (2022) Dialoglm: pre-trained model for long dialogue understanding and summarization. Proc AAAI Conf Artif Intell 36:11765–11773

27.

Tay Y, Bahri D, Yang L, Metzler D, Juan D-C (2020) Sparse sinkhorn attention. In: International conference on machine learning, pp. 9438–9447 . PMLR

28.

Zhang Y, Ni A, Mao Z, Wu CH, Zhu C, Deb B, Awadallah AH, Radev D, Zhang R (2022) Summn: A multi-stage summarization framework for long input dialogues and documents. In: 60th annual meeting of the association for computational linguistics, ACL 2022, pp. 1592–1604 . Association for Computational Linguistics (ACL)

29.

Zhao L, Xu W, Guo J (2020) Improving abstractive dialogue summarization with graph structures and topic words. In: Proceedings of the 28th international conference on computational linguistics, pp. 437–449

30.

Feng X, Feng X, Qin B (2021) Incorporating commonsense knowledge into abstractive dialogue summarization via heterogeneous graph networks. In: China national conference on chinese computational linguistics, pp. 127–142 . Springer

31.

Liu Z, Shi K, Chen N (2021) Coreference-aware dialogue summarization. In: Proceedings of the 22nd annual meeting of the special interest group on discourse and dialogue, pp. 509–519

32.

Zhang Y, Sun S, Galley M, Chen Y-C, Brockett C, Gao X, Gao J, Liu J, Dolan B (2020) DIALOGPT : large-scale generative pre-training for conversational response generation. In: Celikyilmaz A, Wen T-H (eds.) Proceedings of the 58th annual meeting of the association for computational linguistics: system demonstrations, pp. 270–278. Association for computational linguistics, Online . https://doi.org/10.18653/v1/2020.acl-demos.30 . https://aclanthology.org/2020.acl-demos.30

33.

Cho K, Merrienboer B, Gulcehre C, Bahdanau D, Bougares F, Schwenk H, Bengio Y (2014) Learning phrase representations using rnn encoder–decoder for statistical machine translation. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pp. 1724–1734

34.

Li M, Zhang L, Ji H, Radke RJ (2019) Keep meeting summaries on topic: Abstractive multi-modal meeting summarization. In: Proceedings of the 57th annual meeting of the association for computational linguistics, pp. 2190–2196

35.

Manakul P, Gales M 2021) Long-span summarization via local attention and content selection. In: Proceedings of the 59th annual meeting of the association for computational linguistics and the 11th international joint conference on natural language processing (Volume 1: Long Papers), pp. 6026–6041 (

36.

Papineni K, Roukos S, Ward T, Zhu W-J (2002) Bleu: a method for automatic evaluation of machine translation. In: Isabelle P, Charniak E, Lin D (eds.) Proceedings of the 40th annual meeting of the association for computational linguistics, pp. 311–318. Association for computational linguistics, Philadelphia, Pennsylvania, USA . https://doi.org/10.3115/1073083.1073135 . https://aclanthology.org/P02-1040

37.

Shleifer S, Rush AM (2020) Pre-trained summarization distillation. arXiv preprint arXiv:2010.13002

38.

Raffel C, Shazeer N, Roberts A, Lee K, Narang S, Matena M, Zhou Y, Li W, Liu PJ (2020) Exploring the limits of transfer learning with a unified text-to-text transformer. J Mach Learn Res 21(140):1–67MathSciNet

39.

Zhang J, Zhao Y, Saleh M, Liu P (2020) Pegasus: pre-training with extracted gap-sentences for abstractive summarization. In: International conference on machine learning, pp. 11328–11339 . PMLR

40.

Lewis M, Liu Y, Goyal N, Ghazvininejad M, Mohamed A, Levy O, Stoyanov V, Zettlemoyer L (2020) BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In: Jurafsky D, Chai J, Schluter N, Tetreault J (eds.) Proceedings of the 58th annual meeting of the association for computational linguistics, pp. 7871–7880. Association for computational linguistics, Online . https://doi.org/10.18653/v1/2020.acl-main.703 . https://aclanthology.org/2020.acl-main.703

41.

Park S, Shin D, Lee J (2022) Leveraging non-dialogue summaries for dialogue summarization. In: Dernoncourt F, Nguyen TH, Lai VD, Veyseh APB, Bui TH, Yoon DS (eds.) Proceedings of the first workshop on transcript understanding, pp. 1–7. International conference on computational linguistics, Gyeongju, South Korea . https://aclanthology.org/2022.tu-1.1

42.

Asi A, Wang S, Eisenstadt R, Geckt D, Kuper Y, Mao Y, Ronen R (2022) An end-to-end dialogue summarization system for sales calls. In: Loukina A, Gangadharaiah R, Min B (eds.) Proceedings of the 2022 conference of the North American chapter of the association for computational linguistics: human language technologies: industry track, pp. 45–53. Association for computational linguistics, Hybrid: Seattle, Washington + Online . https://doi.org/10.18653/v1/2022.naacl-industry.6 . https://aclanthology.org/2022.naacl-industry.6

43.

Xie K, He D, Zhuang J, Lu S, Wang Z (2022) View dialogue in 2D: a two-stream model in time-speaker perspective for dialogue summarization and beyond. In: Calzolari N, Huang C-R, Kim H, Pustejovsky J, Wanner L, Choi K-S, Ryu P-M, Chen H-H, Donatelli L, Ji H, Kurohashi S, Paggio P, Xue N, Kim S, Hahm Y, He Z, Lee TK, Santus E, Bond F, Na S-H (eds.) Proceedings of the 29th international conference on computational linguistics, pp. 6075–6088. International committee on computational linguistics, Gyeongju, Republic of Korea . https://aclanthology.org/2022.coling-1.531

44.

Dubey A, Jauhri A, Pandey A, Kadian A, Al-Dahle A, Letman A, Mathur A, Schelten A, Yang A, Fan A, et al (2024) The llama 3 herd of models. arXiv preprint arXiv:2407.21783

45.

Zhao L, Zeng W, Xu W, Guo J (2021) Give the truth: Incorporate semantic slot into abstractive dialogue summarization. In: Moens M-F, Huang X, Specia L, Yih SW-T (eds.) Findings of the association for computational linguistics: EMNLP 2021, pp. 2435–2446. Association for computational linguistics, Punta Cana, Dominican Republic . https://doi.org/10.18653/v1/2021.findings-emnlp.209 . https://aclanthology.org/2021.findings-emnlp.209

46.

Ravaut M, Joty S, Chen N (2022) Towards summary candidates fusion. In: Goldberg Y, Kozareva Z, Zhang Y (eds.) Proceedings of the 2022 conference on empirical methods in natural language processing, pp. 8488–8504. Association for computational linguistics, Abu Dhabi, United Arab Emirates . https://doi.org/10.18653/v1/2022.emnlp-main.581 . https://aclanthology.org/2022.emnlp-main.581

47.

Liang X, Wu S, Cui C, Bai J, Bian C, Li Z (2023) Enhancing dialogue summarization with topic-aware global- and local- level centrality. In: Vlachos A, Augenstein I (eds.) Proceedings of the 17th conference of the european chapter of the association for computational linguistics, pp. 27–38. Association for computational linguistics, Dubrovnik, Croatia . https://doi.org/10.18653/v1/2023.eacl-main.2 . https://aclanthology.org/2023.eacl-main.2

48.

Duan J, Lu F, Liu J (2024) Prophecy distillation for boosting abstractive summarization. In: Calzolari N, Kan M-Y, Hoste V, Lenci A, Sakti S, Xue N (eds.) Proceedings of the 2024 joint international conference on computational linguistics, language resources and evaluation (LREC-COLING 2024), pp. 13257–13268. ELRA and ICCL, Torino, Italia . https://aclanthology.org/2024.lrec-main.1160

49.

Li Y, Peng B, He P Galley M, Yu Z, Gao J (2023) Dionysus: A pre-trained model for low-resource dialogue summarization. In: Proceedings of the 61st annual meeting of the association for computational linguistics (Volume 1: Long Papers), pp. 1368–1386

50.

Park S, Choi H, Choi D, Lee H (2024) Gendex: generative data augmentation strategy leveraging external data for abstractive dialogue summarization. Find Assoc Comput Linguist ACL 2024:3171–3185MATH

51.

Yoo C, Lee H (2023) Improving abstractive dialogue summarization using keyword extraction. Appl Sci 13(17):9771CrossRef

Title: RDPNet: a multi-stage summary generation network for long chat dialogues
Authors: Shihao Yang
Shaoru Zhang
Huayu Zhang
Chih-Cheng Hung
Fengqin Yang
Shuhua Liu
Publication date: 12-02-2025
Publisher: Springer London
Published in: Knowledge and Information Systems
Print ISSN: 0219-1377
Electronic ISSN: 0219-3116
DOI: https://doi.org/10.1007/s10115-025-02358-w

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Wirtschaft"

Springer Professional "Technik"