Neurocomputing
Align-Then-Abstract Representation Learning for Low-Resource Summarization
Athena
Align-Then-Abstract Representation Learning for Low-Resource Summarization
Gianluca Moro, Luca Ragazzi
Neurocomputing
Description
Generative transformer-based models have achieved state-of-the-art performance in text summarization. Nevertheless, they still struggle in real-world scenarios with long documents when trained in low-resource settings of a few dozen labeled training instances, namely in low-resource summarization (LRS). This paper bridges the gap by addressing two key research challenges when summarizing long documents, i.e., long-input processing and document representation, in one coherent model trained for LRS. Specifically, our novel align-then-abstract representation learning model (Athena) jointly trains a segmenter and a summarizer by maximizing the alignment between the chunk-target pairs in output from the text segmentation. Extensive experiments reveal that Athena outperforms the current state-of-the-art approaches in LRS on multiple long document summarization datasets from different domains.
Keywords: long document summarization, abstractive summarization, low-resource, representation learning, NLP
Code: https://github.com/disi-unibo-nlp/athena
If you use Athena
in your research, please cite Align-Then-Abstract Representation Learning for Low-Resource Summarization.
@article{moro2023align,
title={Align-Then-Abstract Representation Learning for Low-Resource Summarization},
author={Moro, Gianluca and Ragazzi, Luca},
journal={Neurocomputing},
pages={126356},
year={2023},
publisher={Elsevier}
}