Advertisement
Published in:
2017 | OriginalPaper | Chapter
Document Classification Using Word2Vec and Chi-square on Apache Spark
Authors : Mijin Choi, Rize Jin, Tae-Sun Chung
Published in: Advances in Computer Science and Ubiquitous Computing
Publisher: Springer Singapore
Activate our intelligent search to find suitable subject content or patents.
Select sections of text to find matching patents with Artificial Intelligence. powered by
Select sections of text to find additional relevant content using AI-assisted search. powered by
Abstract
Text mining is a mechanism to find information by extracting resources from natural language. Compared with structured data in databases, text is unstructured and difficult to be dealt with for analyzing. Additionally, it is tedious tasks for users to identify accurate data. Text mining algorithm is similar to data mining, except that it processes data in database and aims to determine whether any document belongs to a specific topic. There are some classification algorithms. To identify which classifier is efficient, we compare SVM (Support Vector Machine) and Naïve Bayes, and use Apache Spark which is distributed system environment, to classify a large number of documents efficiently.