The Summary Loop: Learning to Write Abstractive Summaries Without Examples

This work presents a new approach to unsupervised abstractive summarization\nbased on maximizing a combination of coverage and fluency for a given length\nconstraint. It introduces a novel method that encourages the inclusion of key\nterms from the original document into the summary: key terms are masked out of\nthe original document and must be filled in by a coverage model using the\ncurrent generated summary. A novel unsupervised training procedure leverages\nthis coverage model along with a fluency model to generate and score summaries.\nWhen tested on popular news summarization datasets, the method outperforms\nprevious unsupervised methods by more than 2 R-1 points, and approaches results\nof competitive supervised methods. Our model attains higher levels of\nabstraction with copied passages roughly two times shorter than prior work, and\nlearns to compress and merge sentences without supervision.\n