Update README.md
Browse files
README.md
CHANGED
@@ -22,10 +22,10 @@ model-index:
|
|
22 |
type: squad_it
|
23 |
metrics:
|
24 |
- type: exact-match
|
25 |
-
value: 0.
|
26 |
name: Exact Match
|
27 |
- type: f1
|
28 |
-
value: 0.
|
29 |
name: F1
|
30 |
pipeline_tag: question-answering
|
31 |
---
|
@@ -63,11 +63,23 @@ The 2.0 version further improves the performances by exploiting a 2-phases fine-
|
|
63 |
|
64 |
In order to maximize the benefits of the multilingual procedure, [mdeberta-v3-base](https://huggingface.co/microsoft/mdeberta-v3-base) is used as a pre-trained model. When the double fine-tuning is completed, the embedding layer is then compressed as in [deberta-base-italian](https://huggingface.co/osiria/deberta-base-italian) to obtain a mono-lingual model size
|
65 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
66 |
<b>update: version 3.0 (LITEQA)</b>
|
67 |
|
68 |
The 3.0 version, with the nickname LITEQA, further improves the performances by exploiting a 3-phases fine-tuning strategy: the model is first fine-tuned on the English SQuAD v2 (1 epoch, 20% warmup ratio, and max learning rate of 3e-5) then further fine-tuned on the Italian SQuAD (2 epochs, no warmup, initial learning rate of 3e-5) and lastly fine-tuned on the lowercase Italian SQuAD (1 epochs, no warmup, initial learning rate of 3e-5)
|
69 |
This helps making the model generally more robust, but particularly in uncased settings.
|
70 |
|
|
|
|
|
71 |
The performances on the test set are reported in the following table:
|
72 |
|
73 |
(<b>version 3.0</b> performances)
|
|
|
22 |
type: squad_it
|
23 |
metrics:
|
24 |
- type: exact-match
|
25 |
+
value: 0.7004
|
26 |
name: Exact Match
|
27 |
- type: f1
|
28 |
+
value: 0.8097
|
29 |
name: F1
|
30 |
pipeline_tag: question-answering
|
31 |
---
|
|
|
63 |
|
64 |
In order to maximize the benefits of the multilingual procedure, [mdeberta-v3-base](https://huggingface.co/microsoft/mdeberta-v3-base) is used as a pre-trained model. When the double fine-tuning is completed, the embedding layer is then compressed as in [deberta-base-italian](https://huggingface.co/osiria/deberta-base-italian) to obtain a mono-lingual model size
|
65 |
|
66 |
+
The performances on the test set are reported in the following table:
|
67 |
+
|
68 |
+
(<b>version 2.0</b> performances)
|
69 |
+
|
70 |
+
| EM | F1 |
|
71 |
+
| ------ | ------ |
|
72 |
+
| 70.04 | 80.97 |
|
73 |
+
|
74 |
+
Testing notebook: https://huggingface.co/osiria/deberta-italian-question-answering/blob/main/osiria_deberta_italian_qa_evaluation.ipynb
|
75 |
+
|
76 |
<b>update: version 3.0 (LITEQA)</b>
|
77 |
|
78 |
The 3.0 version, with the nickname LITEQA, further improves the performances by exploiting a 3-phases fine-tuning strategy: the model is first fine-tuned on the English SQuAD v2 (1 epoch, 20% warmup ratio, and max learning rate of 3e-5) then further fine-tuned on the Italian SQuAD (2 epochs, no warmup, initial learning rate of 3e-5) and lastly fine-tuned on the lowercase Italian SQuAD (1 epochs, no warmup, initial learning rate of 3e-5)
|
79 |
This helps making the model generally more robust, but particularly in uncased settings.
|
80 |
|
81 |
+
The 3.0 version can be downloaded from the <b>liteqa<b> branch of this repo
|
82 |
+
|
83 |
The performances on the test set are reported in the following table:
|
84 |
|
85 |
(<b>version 3.0</b> performances)
|