Learning Interactions at Multiple Levels for Abstractive Multi-document Summarization

The biggest obstacles facing multi-document summarization include much more complicated input and excessive redundancy in source contents. Most state-of-the-art systems have attempted to tackle the redundancy problem, treating the entire input as a flat sequence. However, correlations among documents are often neglected. In this paper, we propose an end-to-end summarization model called MLT, which can effectively learn interactions at multiple levels and avoid redundant information. Specifically, we utilize a word-level transformer layer to encode contextual information within each sentence. Also, we design a sentence-level transformer layer for learning relations between sentences within a single document, as well as a document-level layer for learning interactions among input documents. Moreover, we use a neural method to enhance Max Marginal Relevance (MMR), a powerful algorithm for redundancy reduction. We incorporate MMR into our model and measure the redundancy quantitively based on the sentence representations. On benchmark datasets, our system compares favorably to strong summarization baselines judged by automatic metrics and human evaluators.