Update README.md
Browse files
README.md
CHANGED
@@ -38,6 +38,11 @@ More information needed
|
|
38 |
## Training procedure
|
39 |
|
40 |
- Preprocessing function was created to tokenize the text and truncate the sequences longer than DistilBERT max seq length. Datasets [map](https://huggingface.co/docs/datasets/main/en/package_reference/main_classes#datasets.Dataset.map) function was used to apply the preprocessing func over the entire dataset.
|
|
|
|
|
|
|
|
|
|
|
41 |
### Training hyperparameters
|
42 |
|
43 |
The following hyperparameters were used during training:
|
|
|
38 |
## Training procedure
|
39 |
|
40 |
- Preprocessing function was created to tokenize the text and truncate the sequences longer than DistilBERT max seq length. Datasets [map](https://huggingface.co/docs/datasets/main/en/package_reference/main_classes#datasets.Dataset.map) function was used to apply the preprocessing func over the entire dataset.
|
41 |
+
- DataCollatorWithPadding is more efficiently method to dynamically pad the sequences to the longest length in a batch during collation, instead of padding the whole dataset to the maximum length.
|
42 |
+
- To evaluate model performance during training, it's quite helpful to include a metric. Load the [accuracy](https://huggingface.co/spaces/evaluate-metric/accuracy) metric from [Evaluate](https://huggingface.co/docs/evaluate/index) library.
|
43 |
+
- Define training hyperparameters in [TrainingArguments](https://huggingface.co/docs/transformers/main/en/main_classes/trainer#transformers.TrainingArguments). To push the model to the hub, we need to set `push_to_hub=True`. At the end of each epoch, the [Trainer](https://huggingface.co/docs/transformers/main/en/main_classes/trainer#transformers.Trainer) will evaluate the accuracy and save the training checkpoint.
|
44 |
+
- Pass the trainingargs to the Trainer along with the model, dataset, tokenizer, data_collator.
|
45 |
+
- Call [train()](https://huggingface.co/docs/transformers/main/en/main_classes/trainer#transformers.Trainer.train) to finetune your model.
|
46 |
### Training hyperparameters
|
47 |
|
48 |
The following hyperparameters were used during training:
|