Tanvi03 commited on
Commit
a011ae9
·
verified ·
1 Parent(s): 5a6ce06

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +15 -16
README.md CHANGED
@@ -43,12 +43,13 @@ ReidLM, like all large language models, has inherent biases and limitations that
43
 
44
  This section is meant to convey recommendations with respect to the bias, risk, and technical limitations.
45
 
46
- Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.--->
47
 
48
  ## Getting Started with the Model
49
 
50
  Use the code below to get started with the model.
51
  ## Use with Transformers AutoModelForCausalLM
 
52
  ```
53
  import transformers
54
  import torch
@@ -69,11 +70,9 @@ generated_text = generate_text(prompt)
69
  print(generated_text)
70
  ```
71
 
72
- <br>
73
-
74
  ## Training Details
75
 
76
-
77
  ### Training Data
78
 
79
  <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
@@ -86,18 +85,18 @@ print(generated_text)
86
 
87
  #### Training Hyperparameters
88
 
89
- num_train_epochs=3, <br>
90
- per_device_train_batch_size=4,<br>
91
- gradient_accumulation_steps=2,<br>
92
- optim="paged_adamw_8bit",<br>
93
- save_steps=1000,<br>
94
- logging_steps=30,<br>
95
- learning_rate=2e-4,<br>
96
- weight_decay=0.01,<br>
97
- fp16=True,<br>
98
- max_grad_norm=1.0,<br>
99
- warmup_ratio=0.1<br><!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
100
-
101
  <!---#### Speeds, Sizes, Times [optional]
102
 
103
  <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
 
43
 
44
  This section is meant to convey recommendations with respect to the bias, risk, and technical limitations.
45
 
46
+ Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.-->
47
 
48
  ## Getting Started with the Model
49
 
50
  Use the code below to get started with the model.
51
  ## Use with Transformers AutoModelForCausalLM
52
+
53
  ```
54
  import transformers
55
  import torch
 
70
  print(generated_text)
71
  ```
72
 
 
 
73
  ## Training Details
74
 
75
+ <!-- -->
76
  ### Training Data
77
 
78
  <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
 
85
 
86
  #### Training Hyperparameters
87
 
88
+ num_train_epochs=3, <br>
89
+ per_device_train_batch_size=4,<br>
90
+ gradient_accumulation_steps=2,<br>
91
+ optim="paged_adamw_8bit",<br>
92
+ save_steps=1000,<br>
93
+ logging_steps=30,<br>
94
+ learning_rate=2e-4,<br>
95
+ weight_decay=0.01,<br>
96
+ fp16=True,<br>
97
+ max_grad_norm=1.0,<br>
98
+ warmup_ratio=0.1<br>
99
+ <!-- -->
100
  <!---#### Speeds, Sizes, Times [optional]
101
 
102
  <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->