sultan commited on
Commit
b93b554
1 Parent(s): 6f6d2dc

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +12 -3
README.md CHANGED
@@ -50,10 +50,13 @@ You can download the full details of our grid search for all models in all tasks
50
 
51
  For the XL-Sum task, we choose our best run for each model using the eval set. We use the official evaluation script from XL-Sum, which uses the stemmer function, which may show better results than papers that don't use the stemmer function. The official XL-Sum paper uses a stemmer function.
52
 
 
 
 
 
 
 
53
 
54
- # Continual Pre-Training of ArabicT5 with T5x
55
- if you want to continue pre-training ArabicT5 on your own data, we have uploaded the raw t5x checkpoint to this link https://huggingface.co/sultan/ArabicT5-49GB-base/blob/main/arabict5_49GB_base_t5x.tar.gz
56
- We will soon share a tutorial on how you can do that for free with Kaggle TPU
57
 
58
 
59
 
@@ -63,6 +66,12 @@ We will soon share a tutorial on how you can do that for free with Kaggle TPU
63
 
64
  [COLAB]: https://colab.research.google.com/assets/colab-badge.svg
65
 
 
 
 
 
 
 
66
  ## GitHub Page
67
 
68
  https://github.com/salrowili/ArabicT5
 
50
 
51
  For the XL-Sum task, we choose our best run for each model using the eval set. We use the official evaluation script from XL-Sum, which uses the stemmer function, which may show better results than papers that don't use the stemmer function. The official XL-Sum paper uses a stemmer function.
52
 
53
+ # FineTuning our efficient ArabicT5-49GB-Small model with Torch on 3070 laptop GPU ###
54
+
55
+ If you are running your code on a laptop GPU (e.g., a gaming laptop) or limited GPU memory, we recommended using our ArabicT5-49GB-Small model, which was the only model from the list that we were able to run on 3070 Laptop card with a batch size of 8. We manage to achieve an F1 score of 85.391 (slightly better than our FLAX code ) on the TyDi QA task. See the notebook below for reference :
56
+
57
+ [![Open In Colab][COLAB]](https://colab.research.google.com/github/salrowili/ArabicT5/blob/main/ArabicT5_49GB_Small_on_3070_Laptop_GPU.ipynb)
58
+
59
 
 
 
 
60
 
61
 
62
 
 
66
 
67
  [COLAB]: https://colab.research.google.com/assets/colab-badge.svg
68
 
69
+
70
+ # Continual Pre-Training of ArabicT5 with T5x
71
+ if you want to continue pre-training ArabicT5 on your own data, we have uploaded the raw t5x checkpoint to this link https://huggingface.co/sultan/ArabicT5-49GB-base/blob/main/arabict5_49GB_base_t5x.tar.gz
72
+ We will soon share a tutorial on how you can do that for free with Kaggle TPU
73
+
74
+
75
  ## GitHub Page
76
 
77
  https://github.com/salrowili/ArabicT5