abertsch/unlimiformer-bart-govreport-alternating

Model from the preprint Unlimiformer: Long-Range Transformers with Unlimited Length Input

This is a BART-base model finetuned using the Unlimiformer alternating-training method, as described in section 3.2 of the paper. The model was finetuned on GovReport using the data processing pipeline from SLED; to load the validation or test set for use with these model, please use the datasets urialon/gov_report_validation and urialon/gov_report_test.

This is the strongest of the Unlimiformer models on this dataset.

The inference demo is disabled because you must add the Unlimiformer files to your repo before this model can handle unlimited length input! See the Unlimiformer GitHub for setup instructions.