Update README.md
Browse files
README.md
CHANGED
@@ -50,13 +50,13 @@ pipeline_tag: question-answering
|
|
50 |
|
51 |
<h3>Model description</h3>
|
52 |
|
53 |
-
This is a <b>DeBERTa</b> <b>[1]</b> model for the <b>Italian</b> language, fine-tuned for <b>Extractive Question Answering</b> on the [SQuAD-IT](https://huggingface.co/datasets/squad_it) dataset <b>[2]</b
|
54 |
|
55 |
<b>update: version 2.0</b>
|
56 |
|
57 |
The 2.0 version further improves the performances by exploiting a 2-phases fine-tuning strategy: the model is first fine-tuned on the English SQuAD v2 (1 epoch, 20% warmup ratio, and max learning rate of 3e-5) then further fine-tuned on the Italian SQuAD (2 epochs, no warmup, initial learning rate of 3e-5)
|
58 |
|
59 |
-
In order to maximize the benefits of the procedure, [mdeberta-v3-base](https://huggingface.co/microsoft/mdeberta-v3-base) is
|
60 |
|
61 |
<h3>Training and Performances</h3>
|
62 |
|
|
|
50 |
|
51 |
<h3>Model description</h3>
|
52 |
|
53 |
+
This is a <b>DeBERTa</b> <b>[1]</b> model for the <b>Italian</b> language, fine-tuned for <b>Extractive Question Answering</b> on the [SQuAD-IT](https://huggingface.co/datasets/squad_it) dataset <b>[2]</b>.
|
54 |
|
55 |
<b>update: version 2.0</b>
|
56 |
|
57 |
The 2.0 version further improves the performances by exploiting a 2-phases fine-tuning strategy: the model is first fine-tuned on the English SQuAD v2 (1 epoch, 20% warmup ratio, and max learning rate of 3e-5) then further fine-tuned on the Italian SQuAD (2 epochs, no warmup, initial learning rate of 3e-5)
|
58 |
|
59 |
+
In order to maximize the benefits of the multilingual procedure, [mdeberta-v3-base](https://huggingface.co/microsoft/mdeberta-v3-base) is used as a pre-trained model. When the double fine-tuning is completed, the embedding layer is then compressed as in [deberta-base-italian](https://huggingface.co/osiria/deberta-base-italian) to obtain a mono-lingual model size
|
60 |
|
61 |
<h3>Training and Performances</h3>
|
62 |
|