5. Improving Sentence Extraction Through Rank Aggregation

A plethora of extractive summarisation techniques have been developed in the past decade, but very few enquiries have been made as to how these differ from each other or what factors affect these systems. Such meaningful comparison if available can be used to create a robust ensemble of these approaches, which has the possibility to consistently outperform each individual summarisation system. In this chapter we examine the roles of three principle components of an extractive summarisation technique: sentence ranking algorithm, sentence similarity metric and text representation scheme. We show that using a combination of several different sentence similarity measures, rather than choosing any particular measure, significantly improves performance of the resultant meta-system. Even simple ensemble techniques, when used in an informed manner, prove to be very effective in improving the overall performance and consistency of summarisation systems. While aggregating multiple ranking algorithms or text similarity measures, though the improvement in ROUGE score is not always significant, the resultant meta-systems are more robust than candidate systems. The results suggest that, when proposing a sentence extraction technique, defining better sentence similarity metrics would be more impactful than a new ranking algorithm. Also using multiple sentence similarity scores and ranking algorithms in favour of a particular combination always results in an improved and robust performance.