Athena

Neurocomputing

Generative transformer-based models have achieved state-of-the-art performance in text summarization. Nevertheless, they still struggle in real-world scenarios with long documents when trained in low-resource settings of a few dozen labeled training instances, namely in low-resource summarization (LRS). This paper bridges the gap by addressing two key research challenges when summarizing long documents, i.e., long-input processing and document representation, in one coherent model trained for LRS. Specifically, our novel align-then-abstract representation learning model (Athena) jointly trains a segmenter and a summarizer by maximizing the alignment between the chunk-target pairs in output from the text segmentation. Extensive experiments reveal that Athena outperforms the current state-of-the-art approaches in LRS on multiple long document summarization datasets from different domains.

Keywords: long document summarization, abstractive summarization, low-resource, representation learning, NLP

Code: https://github.com/disi-unibo-nlp/athena

Citing

If you use Athena in your research, please cite Align-Then-Abstract Representation Learning for Low-Resource Summarization.

@article{moro2023align,
  title={Align-Then-Abstract Representation Learning for Low-Resource Summarization},
  author={Moro, Gianluca and Ragazzi, Luca},
  journal={Neurocomputing},
  pages={126356},
  year={2023},
  publisher={Elsevier}
}