pszemraj commited on
Commit
8947be6
·
1 Parent(s): 7e2f082

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +6 -1
README.md CHANGED
@@ -7,4 +7,9 @@ sdk: static
7
  pinned: false
8
  ---
9
 
10
- Edit this `README.md` markdown file to author your organization card 🔥
 
 
 
 
 
 
7
  pinned: false
8
  ---
9
 
10
+ # stacked summaries
11
+
12
+
13
+ This organization exists to test and evaluate the (_potential_) benefits of "task-oriented pretraining" as popularized by the [FLAN-t5](https://huggingface.co/google/flan-t5-base) series of models
14
+
15
+ The idea is to apply a similar concept but adjusted to be more specific w.r.t. the summarization task. Hopefully, this will train models that actually "know" how to condense and distill meaningful information from text rather than learning some naive style transfer of "this is what the dataset summaries sound like." The most apparent augmentation/task is "stacking" summaries that are shorter than `MAX_LENGTH_TOKENS`, so the model has to learn to separate and group summaries for these independent concepts.