Commit
•
ef02c99
1
Parent(s):
7fe2ad3
Update README.md
Browse files
README.md
CHANGED
|
@@ -57,7 +57,7 @@ This is the [deberta-v3-base](https://huggingface.co/microsoft/deberta-v3-base)
|
|
| 57 |
**Eval data:** SQuAD 2.0
|
| 58 |
**Infrastructure**: 1x NVIDIA 3070
|
| 59 |
|
| 60 |
-
|
| 61 |
```python
|
| 62 |
from transformers import AutoModelForQuestionAnswering, AutoTokenizer, pipeline
|
| 63 |
model_name = "sjrhuschlee/deberta-v3-base-squad2"
|
|
@@ -75,7 +75,7 @@ model = AutoModelForQuestionAnswering.from_pretrained(model_name)
|
|
| 75 |
tokenizer = AutoTokenizer.from_pretrained(model_name)
|
| 76 |
```
|
| 77 |
|
| 78 |
-
|
| 79 |
|
| 80 |
```bash
|
| 81 |
# Squad v2
|
|
@@ -108,4 +108,27 @@ tokenizer = AutoTokenizer.from_pretrained(model_name)
|
|
| 108 |
"eval_samples_per_second": 54.392,
|
| 109 |
"eval_steps_per_second": 2.269
|
| 110 |
}
|
| 111 |
-
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 57 |
**Eval data:** SQuAD 2.0
|
| 58 |
**Infrastructure**: 1x NVIDIA 3070
|
| 59 |
|
| 60 |
+
## Model Usage
|
| 61 |
```python
|
| 62 |
from transformers import AutoModelForQuestionAnswering, AutoTokenizer, pipeline
|
| 63 |
model_name = "sjrhuschlee/deberta-v3-base-squad2"
|
|
|
|
| 75 |
tokenizer = AutoTokenizer.from_pretrained(model_name)
|
| 76 |
```
|
| 77 |
|
| 78 |
+
## Metrics
|
| 79 |
|
| 80 |
```bash
|
| 81 |
# Squad v2
|
|
|
|
| 108 |
"eval_samples_per_second": 54.392,
|
| 109 |
"eval_steps_per_second": 2.269
|
| 110 |
}
|
| 111 |
+
```
|
| 112 |
+
|
| 113 |
+
## Training procedure
|
| 114 |
+
|
| 115 |
+
### Training hyperparameters
|
| 116 |
+
|
| 117 |
+
The following hyperparameters were used during training:
|
| 118 |
+
- learning_rate: 5e-06
|
| 119 |
+
- train_batch_size: 8
|
| 120 |
+
- eval_batch_size: 8
|
| 121 |
+
- seed: 42
|
| 122 |
+
- gradient_accumulation_steps: 8
|
| 123 |
+
- total_train_batch_size: 64
|
| 124 |
+
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
|
| 125 |
+
- lr_scheduler_type: linear
|
| 126 |
+
- lr_scheduler_warmup_ratio: 0.1
|
| 127 |
+
- num_epochs: 4.0
|
| 128 |
+
|
| 129 |
+
### Framework versions
|
| 130 |
+
|
| 131 |
+
- Transformers 4.30.0.dev0
|
| 132 |
+
- Pytorch 2.0.1+cu117
|
| 133 |
+
- Datasets 2.12.0
|
| 134 |
+
- Tokenizers 0.13.3
|