31. Classifying Sincerity Using Machine Learning

Quora is an online platform that empowers people to learn from each other. On Quora, users can post questions and connect with others who contribute unique insights and quality answers. But as with any other social media or online platform, there is the potential for misuse. A key challenge in maintaining the integrity of such an online platform is to classify and flag negative content. On Quora, the challenge is to identify insincere questions. Insincere questions could be those founded upon false premises, are disparaging, inflammatory, intended to arouse discrimination in any form, or intend to make a statement rather than look for helpful answers. We propose to develop a text classification model that correctly labels questions as sincere or insincere on the Quora platform. For this purpose, we used the Quora Insincere Questions Classification dataset, which is available on Kaggle. We first trained classical machine learning models such as Logistic regression and SVMs to establish a baseline on the performance. However, to leverage the large dataset, we used neural network-based models. We trained several models including standard neural networks, and LSTM based models. The best model that we obtained is a two-layer Bidirectional LSTM network that takes word embeddings as inputs. The classification accuracy and F1-score for this model were 96% and 0.64, respectively.