Update README.md
Browse files
README.md
CHANGED
@@ -121,7 +121,7 @@ We finetuned the base model (flan-t5-large) on multiple relevant tasks with stan
|
|
121 |
| Recontextualization | Editor Abstract | "contextualize: " | 2,200 |
|
122 |
| Simplification | Wiki Auto | "simplify: " | 57,000 |
|
123 |
| Summarization | CNN/DailyMail | "summarize: " | 165,000 |
|
124 |
-
| Total | Challenge-proportional
|
125 |
|
126 |
|
127 |
- Multi-instruction tuning: In the stage, we first created a task mixture using "challenge-proportional mixing" method. In a seperate pilot studie, for each task, we finetuned it on a base model and observed the number of samples when validation loss starts to rise. We mixed the samples of each task proportional to its optimal number of samples. A corpus is exhausted before upsampling if the number of total samples is smaller than its optimal number. We finetune with the task mixture (263,400 samples) with the aforementioned template.
|
|
|
121 |
| Recontextualization | Editor Abstract | "contextualize: " | 2,200 |
|
122 |
| Simplification | Wiki Auto | "simplify: " | 57,000 |
|
123 |
| Summarization | CNN/DailyMail | "summarize: " | 165,000 |
|
124 |
+
| Total | Challenge-proportional Mixing | n/a | 263,400 |
|
125 |
|
126 |
|
127 |
- Multi-instruction tuning: In the stage, we first created a task mixture using "challenge-proportional mixing" method. In a seperate pilot studie, for each task, we finetuned it on a base model and observed the number of samples when validation loss starts to rise. We mixed the samples of each task proportional to its optimal number of samples. A corpus is exhausted before upsampling if the number of total samples is smaller than its optimal number. We finetune with the task mixture (263,400 samples) with the aforementioned template.
|