Top

Neural Computing and Applications

Published in:

03-12-2019 | Deep Learning & Neural Computing for Intelligent Sensing and Control

Deep Refinement: capsule network with attention mechanism-based system for text classification

Authors: Deepak Kumar Jain, Rachna Jain, Yash Upadhyay, Abhishek Kathuria, Xiangyuan Lan

Published in: Neural Computing and Applications | Issue 7/2020

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

Most of the text in the questions of community question–answering systems does not consist of a definite mechanism for the restriction of inappropriate and insincere content. A given piece of text can be insincere if it asserts false claims or assumes something which is debatable or has a non-neutral or exaggerated tone about an individual or a group. In this paper, we propose a pipeline called Deep Refinement which utilizes some of the state-of-the-art methods for information retrieval from highly sparse data such as capsule network and attention mechanism. We have applied the Deep Refinement pipeline to classify the text primarily into two categories, namely sincere and insincere. Our novel approach ‘Deep Refinement’ provides a system for the classification of such questions in order to ensure enhanced monitoring and information quality. The database used to understand the real concept of what actually makes up sincere and insincere includes quora insincere question dataset. Our proposed question classification method outperformed previously used text classification methods, as evident from the F1 score of 0.978.

previous article An emotion classification algorithm based on SPT-CapsNet

next article Application research of improved genetic algorithm based on machine learning in production scheduling

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Liu B (2012) Sentiment analysis and opinion mining. Synth Lect Hum Lang Technol 5(1):1–167CrossRef

Novozhilov D, Kotenko I, Chechulin A (2016) Improving the categorization of web sites by analysis of html-tags statistics to block inappropriate content. In: Intelligent distributed computing IX. Springer, Berlin, pp 257–263

Belinkov Y, Mohtarami M, Cyphers S, Glass J (2015) Vectorslu: a continuous word vector approach to answer selection in community question answering systems. In: Proceedings of the 9th international workshop on semantic evaluation (SemEval 2015), pp 282–287

Gabbard S, Yang J, Liu J (2018) Quora insincere question classification. Baskin Engineering, University of California, Santa Cruz

Smith LN (2017) Cyclical learning rates for training neural networks. In: 2017 IEEE winter conference on applications of computer vision (WACV), IEEE, pp 464–472

Wang Z-Q, Sun X, Zhang D-X, Li X (2006) An optimal SVM-based text classification algorithm. In: 2006 International conference on machine learning and cybernetics, IEEE, pp 1378–1381

Liu Z, Lv X, Liu K, Shi S (2010) Study on svm compared with the other text classification methods. In: 2010 Second international workshop on education technology and computer science, vol 1, pp 219–222, IEEE

Haryanto AW, Mawardi EK et al (2018) Influence of word normalization and chi squared feature selection on support vector machine (svm) text classification. In: 2018 International seminar on application for technology of information and communication, IEEE, pp 229–233

Huang Z, Thint M, Qin Z (2008) Question classification using head words and their hypernyms. In: Proceedings of the conference on empirical methods in natural language processing, association for computational linguistics, pp 927–936

10.

Haniewicz K, Rutkowski W, Adamczyk M, Kaczmarek M (2013) Towards the lexicon-based sentiment analysis of polish texts: polarity lexicon. In: International conference on computational collective intelligence. Springer, pp 286–295

11.

Zhang H, Wei H, Tang Y, Pu Q (2019) Research on classification of scientific and technological documents based on naive Bayes. In: Proceedings of the 2019 11th international conference on machine learning and computing. ACM, pp 327–331

12.

Qiang G (2010) An effective algorithm for improving the performance of naive Bayes for text classification. In: 2010 second international conference on computer research and development

13.

Narayanan V, Arora I, Bhatia A (2013) Fast and accurate sentiment classification using an enhanced naive Bayes model. In: International conference on intelligent data engineering and automated learning. Springer, pp 194–201

14.

Pratama BY, Sarno R (2015) Personality classification based on twitter text using naive Bayes, KNN and SVM. In: 2015 International conference on data and software engineering (ICoDSE), IEEE, pp 170–174

15.

Santos CND, Xiang B, Zhou B Classifying relations by ranking with convolutional neural networks. arxiv preprint arXiv:1504.06580

16.

Georgakopoulos SV, Tasoulis SK, Vrahatis AG, Plagianakos VP (2018) Convolutional neural networks for toxic comment classification. In: Proceedings of the 10th Hellenic conference on artificial intelligence. ACM, p 35

17.

Kim Y Convolutional neural networks for sentence classification. arXiv preprint arXiv:1408.5882

18.

Dachapally PR, Ramanam S In-depth question classification using convolutional neural networks. arxiv preprint arXiv:1804.00968

19.

Zhou C, Sun C, Liu Z, Lau F A c-lstm neural network for text classification. arXiv preprint arXiv:1511.08630

20.

Yenala H, Jhanwar A, Chinnakotla MK, Goyal J (2018) Deep learning for detecting inappropriate content in text. Int J Data Sci Anal 6(4):273–286CrossRef

21.

Sabour S, Frosst N, Hinton G (2018) Matrix capsules with EM routing. In: 6th international conference on learning representations, ICLR

22.

Sabour S, Frosst N, Hinton GE (2017) Dynamic routing between capsules. In: Advances in neural information processing systems, pp 3856–3866

23.

Zhao W, Ye J, Yang M, Lei Z, Zhang S, Zhao Z Investigating capsule networks with dynamic routing for text classification. arXiv preprint arXiv:1804.00538

24.

Yang M, Zhao W, Chen L, Qu Q, Zhao Z, Shen Y (2019) Investigating the transferring capability of capsule networks for text classification. Neural Netw 118:247–261CrossRef

25.

Zhao W, Peng H, Eger S, Cambria E, Yang M Towards scalable and reliable capsule networks for challenging NLP applications. arXiv preprint arXiv:1906.02829

26.

Zhang N, Deng S, Sun Z, Chen X, Zhang W, Chen H Attention-based capsule networks with dynamic routing for relation extraction. arXiv preprint arXiv:1812.11321

27.

Li J, Yang B, Dou Z-Y, Wang X, Lyu MR, Tu Z Information aggregation for multi-head attention with routing-by-agreement. arXiv preprint arXiv:1904.03100

28.

Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I (2017) Attention is all you need. In: Advances in neural information processing systems, pp 5998–6008

29.

Mungekar A, Parab N, Nima P, Pereira S (2019) Quora insincere question classification. National College of Ireland

30.

Chen S, Song B, Guo J (2018) Attention alignment multimodal LSTM for fine-gained common space learning. IEEE Access 6:20195–20208CrossRef

31.

Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780CrossRef

32.

Zhou J, Lu Y, Dai HN, Wang H, Xiao H (2019) Sentiment analysis of Chinese microblog based on stacked bidirectional LSTM. IEEE Access 7:38856–38866CrossRef

33.

Long F, Zhou K, Ou W (2019) Sentiment analysis of text based on bidirectional LSTM with multi-head attention. IEEE Access

34.

Bin Y, Yang Y, Shen F, Xie N, Shen HT, Li X (2018) Describing video with attention-based bidirectional LSTM. IEEE Trans Cybern 49(7):2631–2641CrossRef

35.

Kowsari K, Brown DE, Heidarysafa M, Meimandi KJ, Gerber MS, Barnes LE (2017) Hdltex: hierarchical deep learning for text classification. In: 2017 16th IEEE international conference on machine learning and applications (ICMLA), IEEE, pp 364–371

36.

Bahdanau D, Cho K, Bengio Y Neural machine translation by jointly learning to align and translate. arxiv preprint arXiv:1409.0473

37.

Lin A, Li J, Ma Z (2019) On learning and learned data representation by capsule networks. IEEE Access 7:50808–50822CrossRef

38.

Li S, Li M, Xu Y, Bao Z, Fu L, Zhu Y (2018) Capsules based Chinese word segmentation for ancient Chinese medical books. IEEE Access 6:70874–70883CrossRef

39.

Paoletti ME, Haut JM, Fernandez-Beltran R, Plaza J, Plaza A, Li J, Pla F (2018) Capsule networks for hyperspectral image classification. IEEE Trans Geosci Remote Sens 57(4):2145–2160CrossRef

40.

Keren G, Sabato S, Schuller B (2018) Fast single-class classification and the principle of logit separation. In: 2018 IEEE international conference on data mining (ICDM). IEEE, pp 227–236

41.

Rodriguez JD, Perez A, Lozano JA (2009) Sensitivity analysis of k-fold cross validation in prediction error estimation. IEEE Trans Pattern Anal Mach Intell 32(3):569–575CrossRef

42.

Ketkar N (2017) Introduction to pytorch. In: Deep learning with python. Springer, Berlin, pp 195–208

Title: Deep Refinement: capsule network with attention mechanism-based system for text classification
Authors: Deepak Kumar Jain
Rachna Jain
Yash Upadhyay
Abhishek Kathuria
Xiangyuan Lan
Publication date: 03-12-2019
Publisher: Springer London
Published in: Neural Computing and Applications / Issue 7/2020
Print ISSN: 0941-0643
Electronic ISSN: 1433-3058
DOI: https://doi.org/10.1007/s00521-019-04620-z

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Springer Professional "Technik"

Springer Professional "Wirtschaft+Technik"

Other articles of this Issue 7/2020

Classification of sonar echo signals in their reduced sparse forms using complex-valued wavelet neural network

A joint deep neural networks-based method for single nighttime rainy image enhancement

Application research of improved genetic algorithm based on machine learning in production scheduling

Hybrid bio-inspired user clustering for the generation of diversified recommendations

A GA based hierarchical feature selection approach for handwritten word recognition

Automatic optimized support vector regression for financial data prediction

Premium Partner