Transformer-Based Word Embedding With CNN Model to Detect Sarcasm and Irony

Accurate semantic illustrations of text data and conclusive information extraction are major strides towards correct computation of sentence meaning, particularly for figurative languages like humor, irony, and sarcasm. We propose an encoder model called LMTweets, trained on 500 k tweets scraped from Twitter and social media. LMTweets are used to extract the dataset's features, namely SemEval 2018 Task 3. An (Irony), SARC (Sarcasm), and Riloff (Sarcasm). The extracted features are passed as input to the convolution neural network model to classify the text as sarcastic/non-sarcastic and irony/non-irony. We also apply five classification algorithms for the detection of sarcasm/irony, namely Naive Bayes (NB), Logistic Regression (LR), Support Vector Machine (SVM), Random Forest (RF), K-Nearest Neighbors (KNN), six deep learning algorithms, namely Convolutional Neural Network (CNN), Long Short Term Memory (LSTM), Gated Recurrent Unit (GRU), GRU-Pooling, LSTM-Attention Mechanism (AM), GRU-AM and six transformer models namely BERT, RoBERTa, ELECTRA, XLNet, XLM-RoBERTa, and ULMFIT. For the implementation purpose, Keras API is used with Tensorflow as the backend with Python. The performance parameters considered are precision, recall, accuracy, AUC, and f1-score. Experimental results show that LMTweets + CNN model performs better among all models used and gives around 6% better performance on SemEval 2018 Task 3. A dataset, 2–3% on Rillof and SARC datasets shows the results obtained by applying various models are statistically different. The results are validated by applying the ANOVA one-way statistical test.