Automatic clustering of construction project documents based on textual similarity

Highlights

• Hybrid approach for clustering semantically-related project documents is proposed.

• For clustering, inverse relationship between dimensionality & similarity threshold.

• tf–idf weighting method results in high precision, average recall outcomes.

• Refining clustering outcome using supervised learning improves accuracy.

• Textual similarities can be used to reveal semantic relations between documents.

Abstract

Text classifiers, as supervised learning methods, require a comprehensive training set that covers all classes in order to classify new instances. This limits the use of text classifiers for organizing construction project documents since it is not guaranteed that sufficient samples are available for all possible document categories. To overcome the restriction imposed by the all-inclusive requirement, an unsupervised learning method was used to automatically cluster documents together based on textual similarities. Repeated evaluations using different randomizations of the dataset revealed a region of threshold/dimensionality values of consistently high precision values and average recall values. Accordingly, a hybrid approach was proposed which initially uses an unsupervised method to develop core clusters and then trains a text classifier on the core clusters to classify outlier documents in a consequent refinement step. Evaluation of the hybrid approach demonstrated a significant improvement in recall values, resulting in an overall increase in F-measure scores.

Introduction

Automatic classification of documents as a supervised learning method requires a set of class labels and samples of each class in order to conduct the learning process before being able to perform predictions for new document instances. Usually, the classification procedure assumes that the classes are all inclusive (that they make a complete set of all the possible outcomes for any new instance) and that they are mutually exclusive (any new instance can belong to one and only one class). Where classes are static and predefined, the use of text classifiers for automatically organizing documents is appropriate. Documents are traditionally organized in construction projects according to fixed, abstract categories based on document metadata [1]. Examples of studies investigating the use of automatic text classification of construction documents include identifying the corresponding project division for minutes of meeting items [2] and classifying product documents to their relevant division in a construction information classification system [3].

While traditional methods of organizing construction project documents are simple and easy to use, they are not very useful for information retrieval unless the information seeker has thorough knowledge of the document body [1]. Information regarding a researched knowledge topic is almost always distributed over multiple categories thus requiring understanding of document content, not just metadata, to determine relevancy of a document to the researched topic; a time-consuming task that entails the application of human semantic capabilities. Also, the above-mentioned restrictions that constrain the use of classifiers do not apply with unsupervised methods: unsupervised methods do not require previous identification of all possible classes nor are they trained from sample data. The objective of this study is to evaluate the performance of an unsupervised learning text analysis technique in organizing project documents into groups of semantically similar documents; each group defined by its relation to a specific searchable knowledge topic. It is hypothesized that textual similarity between project documents accurately reflects semantic relationships between the documents and, when applied in document management and information retrieval tasks, can achieve results comparable to what humans recognize using their semantic capabilities. In the next section, the text analysis technique used in the study is presented along with several of its applications in previous works. Then the methodology implemented for the evaluation is presented, followed by a detailed analysis of the results. The study is concluded with a summary of the main results and a discussion on practical uses and limitations of implementing the proposed technique.