osiria commited on
Commit
32923d7
1 Parent(s): 09401b3

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +14 -2
README.md CHANGED
@@ -22,10 +22,10 @@ model-index:
22
  type: squad_it
23
  metrics:
24
  - type: exact-match
25
- value: 0.7019
26
  name: Exact Match
27
  - type: f1
28
- value: 0.8101
29
  name: F1
30
  pipeline_tag: question-answering
31
  ---
@@ -63,11 +63,23 @@ The 2.0 version further improves the performances by exploiting a 2-phases fine-
63
 
64
  In order to maximize the benefits of the multilingual procedure, [mdeberta-v3-base](https://huggingface.co/microsoft/mdeberta-v3-base) is used as a pre-trained model. When the double fine-tuning is completed, the embedding layer is then compressed as in [deberta-base-italian](https://huggingface.co/osiria/deberta-base-italian) to obtain a mono-lingual model size
65
 
 
 
 
 
 
 
 
 
 
 
66
  <b>update: version 3.0 (LITEQA)</b>
67
 
68
  The 3.0 version, with the nickname LITEQA, further improves the performances by exploiting a 3-phases fine-tuning strategy: the model is first fine-tuned on the English SQuAD v2 (1 epoch, 20% warmup ratio, and max learning rate of 3e-5) then further fine-tuned on the Italian SQuAD (2 epochs, no warmup, initial learning rate of 3e-5) and lastly fine-tuned on the lowercase Italian SQuAD (1 epochs, no warmup, initial learning rate of 3e-5)
69
  This helps making the model generally more robust, but particularly in uncased settings.
70
 
 
 
71
  The performances on the test set are reported in the following table:
72
 
73
  (<b>version 3.0</b> performances)
 
22
  type: squad_it
23
  metrics:
24
  - type: exact-match
25
+ value: 0.7004
26
  name: Exact Match
27
  - type: f1
28
+ value: 0.8097
29
  name: F1
30
  pipeline_tag: question-answering
31
  ---
 
63
 
64
  In order to maximize the benefits of the multilingual procedure, [mdeberta-v3-base](https://huggingface.co/microsoft/mdeberta-v3-base) is used as a pre-trained model. When the double fine-tuning is completed, the embedding layer is then compressed as in [deberta-base-italian](https://huggingface.co/osiria/deberta-base-italian) to obtain a mono-lingual model size
65
 
66
+ The performances on the test set are reported in the following table:
67
+
68
+ (<b>version 2.0</b> performances)
69
+
70
+ | EM | F1 |
71
+ | ------ | ------ |
72
+ | 70.04 | 80.97 |
73
+
74
+ Testing notebook: https://huggingface.co/osiria/deberta-italian-question-answering/blob/main/osiria_deberta_italian_qa_evaluation.ipynb
75
+
76
  <b>update: version 3.0 (LITEQA)</b>
77
 
78
  The 3.0 version, with the nickname LITEQA, further improves the performances by exploiting a 3-phases fine-tuning strategy: the model is first fine-tuned on the English SQuAD v2 (1 epoch, 20% warmup ratio, and max learning rate of 3e-5) then further fine-tuned on the Italian SQuAD (2 epochs, no warmup, initial learning rate of 3e-5) and lastly fine-tuned on the lowercase Italian SQuAD (1 epochs, no warmup, initial learning rate of 3e-5)
79
  This helps making the model generally more robust, but particularly in uncased settings.
80
 
81
+ The 3.0 version can be downloaded from the <b>liteqa<b> branch of this repo
82
+
83
  The performances on the test set are reported in the following table:
84
 
85
  (<b>version 3.0</b> performances)