Top

The Journal of Supercomputing

Published in:

09-05-2023

An unsupervised opinion summarization model fused joint attention and dictionary learning

Authors: Yu Xiong, Minghe Yan, Xiang Hu, Chaohui Ren, Hang Tian

Published in: The Journal of Supercomputing | Issue 16/2023

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

Unsupervised opinion summarization is the technique of automatically generates summaries without gold reference, and the summaries that reflects aspects of information about the entity. Although there are more mature studies on unsupervised opinion summarazaiton, but these studies focus more on unsupervised training methods and ignore the extraction of information by the model. In this paper, we propose JointSum, an unsupervised opinion summarization method based on variational autoencoder model. JointSum first extracts aspect and sentiment information in reviews by joint attention and dictionary learning, respectively. Joint attention consists of text attention and auxiliary attention, which can extract key information in the input text from different fine-grained levels. Then we calculate the variance and mean of the Gaussian distribution in variational autoencoder model using aspect and sentiment information. In addition, we added the review score prediction subtask to increase the robustness of the model. Finally, in generation phase, we adopt pointer-generator network because it includes copy and coverage mechanism that can solve problems in text generation. Experiments on Amazon and Yelp datasets, the results show that the model has good performance in both automatic and human evaluation, the ROUGE-L value on the Yelp dataset gets 20.83.

next article A GPU-enabled acceleration algorithm for the CAM5 cloud microphysics scheme

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Sun S, Luo C, Chen J (2017) A review of natural language processing techniques for opinion mining systems. Inf Fus 36:10–25CrossRef

Zeng J, Liu T, Jia W, Zhou J (2022) Relation construction for aspect-level sentiment classification. Inf Sci 586:209–223CrossRef

Xu M, Zeng B, Yang H, Chi J, Chen J, Liu H (2022) Combining dynamic local context focus and dependency cluster attention for aspect-level sentiment classification. Neurocomputing 478:49–69CrossRef

Poria S, Cambria E, Gelbukh A (2016) Aspect extraction for opinion mining with a deep convolutional neural network. Knowl-Based Syst 108:42–49CrossRef

Kim, S-M, Hovy E (2004) Determining the sentiment of opinions. In: COLING 2004: Proceedings of the 20th International Conference on Computational Linguistics, pp 1367–1373

Xiao L, Xue Y, Wang H, Hu X, Gu D, Zhu Y (2022) Exploring fine-grained syntactic information for aspect-based sentiment classification with dual graph neural networks. Neurocomputing 471:48–59CrossRef

Mukherjee A, Liu B (2012) Aspect extraction through semi-supervised modeling. In: Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp 339–348

Sohail A, Aslam U, Tariq HI, Jayabalan M (2020) Methodologies and techniques for text summarization: a survey. J Crit Rev 7(11):2020

Isonuma M, Fujino T, Mori J, Matsuo Y, Sakata I (2017) Extractive summarization using multi-task learning with document classification. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp 2101–2110

10.

Song S, Huang H, Ruan T (2019) Abstractive text summarization using LSTM-CNN based deep learning. Multimedia Tools Appl 78(1):857–875CrossRef

11.

Angelidis S, Lapata M (2018) Summarizing opinions: aspect extraction meets sentiment prediction and they are both weakly supervised. arXiv preprint arXiv:1808.08858

12.

Amplayo RK, Lapata M (2020) Unsupervised opinion summarization with noising and denoising. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 1934–1945. Association for Computational Linguistics. https://doi.org/10.18653/v1/2020.acl-main.175. https://aclanthology.org/2020.acl-main.175

13.

Amplayo RK, Angelidis S, Lapata M (2021) Unsupervised opinion summarization with content planning. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol 35, pp 12489–12497

14.

Tošić I, Frossard P (2011) Dictionary learning. IEEE Signal Process Mag 28(2):27–38CrossRefMATH

15.

Xing C, Wu W, Wu Y, Liu J, Huang Y, Zhou M, Ma, W-Y (2017) Topic aware neural response generation. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol 31

16.

Robertson S (2017) Translation with a sequence to sequence network and attention. PyTorch

17.

Woo S, Park J, Lee J-Y, Kweon IS (2018) CBAM: Convolutional block attention module. In: Proceedings of the European Conference on Computer Vision (ECCV), pp 3–19

18.

Aone C (1999) A trainable summarizer with knowledge acquired from robust NLP techniques. In: Advances in automatic text summarization, pp 71–80

19.

Erkan G, Radev DR (2004) LexRank: graph-based lexical centrality as salience in text summarization. J Artif Intell Res 22:457–479CrossRef

20.

Mihalcea R, Tarau P (2004) Textrank: Bringing order into text. In: Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing, pp 404–411

21.

Zheng H, Lapata M (2019) Sentence centrality revisited for unsupervised summarization. arXiv preprint arXiv:1906.03508

22.

Devlin J, Chang M-W, Lee K, Toutanova K (2018) Bert: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805

23.

Rossiello G, Basile P, Semeraro G (2017) Centroid-based text summarization through compositionality of word embeddings. In: Proceedings of the MultiLing 2017 Workshop on Summarization and Summary Evaluation Across Source Types and Genres, pp 12–21

24.

Radford A, Jozefowicz R, Sutskever I (2017) Learning to generate reviews and discovering sentiment. arXiv preprint arXiv:1704.01444

25.

Liu Y (2019) Fine-tune bert for extractive summarization. arXiv preprint arXiv:1903.10318

26.

Ruan Q, Ostendorff M, Rehm G (2022) Histruct+: Improving extractive text summarization with hierarchical structure information. arXiv preprint arXiv:2203.09629

27.

Xie Q, Huang J, Saha T, Ananiadou S (2022) Gretel: Graph contrastive topic enhanced language model for long document extractive summarization. arXiv preprint arXiv:2208.09982

28.

Sutskever I, Vinyals O, Le, QV (2014) Sequence to sequence learning with neural networks. In: Advances in neural information processing systems, vol 27

29.

See A, Liu PJ, Manning CD (2017) Get to the point: Summarization with pointer-generator networks. arXiv preprint arXiv:1704.04368

30.

Nallapati R, Zhou B, Gulcehre C, Xiang B, et al (2016) Abstractive text summarization using sequence-to-sequence RNNs and beyond. arXiv preprint arXiv:1602.06023

31.

Raffel C, Shazeer N, Roberts A, Lee K, Narang S, Matena M, Zhou Y, Li W, Liu PJ (2020) Exploring the limits of transfer learning with a unified text-to-text transformer. J Mach Learn Res 21(1):5485–5551MathSciNet

32.

Liu Y, Lapata M (2019) Hierarchical transformers for multi-document summarization. arXiv preprint arXiv:1905.13164

33.

Li W, Xiao X, Liu J, Wu H, Wang H, Du J (2020) Leveraging graph to improve abstractive multi-document summarization. arXiv preprint arXiv:2005.10043

34.

Laban P, Schnabel T, Bennett PN, Hearst MA (2022) SummaC: Re-visiting NLI-based models for inconsistency detection in summarization. Trans Assoc Comput Linguist 10:163–177CrossRef

35.

Zhao J, Liu M, Gao L, Jin Y, Du L, Zhao H, Zhang H, Haffari G (2020) Summpip: Unsupervised multi-document summarization with sentence graph compression. In: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, pp 1949–1952

36.

Xu S, Zhang X, Wu Y, Wei F, Zhou M (2020) Unsupervised extractive summarization by pre-training hierarchical transformers. arXiv preprint arXiv:2010.08242

37.

Chu E, Liu P (2019) MeanSum: a neural model for unsupervised multi-document abstractive summarization. In: International Conference on Machine Learning, pp 1223–1232. PMLR

38.

Vogler N, Li S, Xu Y, Mi Y, Berg-Kirkpatrick T (2022) An unsupervised masking objective for abstractive multi-document news summarization. arXiv preprint arXiv:2201.02321

39.

Miao Y, Blunsom P (2016) Language as a latent variable: discrete generative models for sentence compression. arXiv preprint arXiv:1609.07317

40.

Bražinskas A, Lapata M, Titov I (2019) Unsupervised opinion summarization as copycat-review generation. arXiv preprint arXiv:1911.02247

41.

Aliakbarpour H, Manzuri MT, Rahmani AM (2022) Improving the readability and saliency of abstractive text summarization using combination of deep neural networks equipped with auxiliary attention mechanism. J Supercomput 78(2):2528–2555CrossRef

42.

He R, McAuley J (2016) Ups and downs: modeling the visual evolution of fashion trends with one-class collaborative filtering. In: Proceedings of the 25th International Conference on World Wide Web, pp 507–517

43.

Wu Y, Schuster M, Chen Z, Le QV, Norouzi M, Macherey W, Krikun M, Cao Y, Gao Q, Macherey K et al (2016) Google’s neural machine translation system: bridging the gap between human and machine translation. arXiv preprint arXiv:1609.08144

44.

Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980

45.

Erkan G, Radev DR (2004) Lexrank: Graph-based lexical centrality as salience in text summarization. Journal of artificial intelligence research 22:457–479

46.

Lin C-Y, Hovy E (2003) Automatic evaluation of summaries using n-gram co-occurrence statistics. In: Proceedings of the 2003 Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics, pp 150–157

47.

Louviere JJ, Woodworth GG (1991) Best-worst scaling: a model for the largest difference judgments. Technical report, Working paper

48.

Louviere JJ, Flynn TN, Marley A (2015) Best-worst scaling: BWS profile case application: preferences for quality of life in Australia 12:240–262. https://doi.org/10.1017/CBO9781107337855

Title: An unsupervised opinion summarization model fused joint attention and dictionary learning
Authors: Yu Xiong
Minghe Yan
Xiang Hu
Chaohui Ren
Hang Tian
Publication date: 09-05-2023
Publisher: Springer US
Published in: The Journal of Supercomputing / Issue 16/2023
Print ISSN: 0920-8542
Electronic ISSN: 1573-0484
DOI: https://doi.org/10.1007/s11227-023-05316-x

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Springer Professional "Technik"

Springer Professional "Wirtschaft+Technik"

Other articles of this Issue 16/2023

CALYOLOv4: lightweight YOLOv4 target detection based on coordinated attention

NoC-based hardware software co-design framework for dataflow thread management

Car depth estimation within a monocular image using a light CNN

Performance evaluation of opportunistic schedulers based on fairness and throughput in new-generation mobile networks

Cuckoo search optimization-based energy efficient job scheduling approach for IoT-edge environment

Survivable SFC deployment method based on federated learning in multi-domain network

Premium Partner