Identification of clickbait news articles using SBERT and correlation matrix

Clickbait refers to the practice of using attention-grabbing or misleading headlines to attract readers to click on particular headlines or pieces of content. This technique often involves exaggerating claims or adding false information to attract traffic. In general, clickbait headlines are unlike standard news posts or detailed articles where headlines are highly correlated with body content. In the proposed system, the dissimilarity between the headline and detailed articles is exploited to create features from headlines and its paragraph by using sentence bidirectional encoder representation from transformer (SBERT). Since the size of the paragraph is quite large compared to the headlines, only k where \(k < l\) (total sentences in a paragraph) sentences from the paragraph are selected using a correlation matrix. The extracted features from the headlines, target title and selected sentences of the paragraph are concatenated and classified using machine learning (ML) classifiers. The proposed model was tested extensively on two real-world datasets, and the results showed that it performed better than the current state-of-the-art models. In experimental results, a support vector machine (SVM) classifier with the concatenated embedding of \(k=6\) dissimilar sentences from paragraph exhibited the best performance with an accuracy of 0.84, weighted precision of 0.83, weighted recall of 0.84, and weighted \(F_1\)-score of 0.82 surpassing the state-of-the-art model by 8.8% in terms of \(F_1\)-score.