macadeliccc
commited on
Update README.md
Browse files
README.md
CHANGED
@@ -13,7 +13,7 @@ pipeline_tag: text-generation
|
|
13 |
---
|
14 |
# Magistrate 3.2 3B
|
15 |
|
16 |
-
Continued pretraining applied to [meta-llama/Llama-3.2-3B](https://huggingface.co/meta-llama/Llama-3.2-3B) using no synthetic data. ~250M tokens.
|
17 |
|
18 |
The model achieves the following results on the evaluation set:
|
19 |
- Loss: 0.6802
|
@@ -210,7 +210,9 @@ More information needed
|
|
210 |
|
211 |
## Training and evaluation data
|
212 |
|
213 |
-
|
|
|
|
|
214 |
|
215 |
## Training procedure
|
216 |
|
|
|
13 |
---
|
14 |
# Magistrate 3.2 3B
|
15 |
|
16 |
+
Continued pretraining applied to [meta-llama/Llama-3.2-3B](https://huggingface.co/meta-llama/Llama-3.2-3B) using no synthetic legal data. ~250M tokens.
|
17 |
|
18 |
The model achieves the following results on the evaluation set:
|
19 |
- Loss: 0.6802
|
|
|
210 |
|
211 |
## Training and evaluation data
|
212 |
|
213 |
+
The training data consists of US Supreme Court verdicts, federal regulations, laws and treaties.
|
214 |
+
|
215 |
+
Some other resources have been included from institutions like CLL to fill in the gaps in knowledge for industry jargon.
|
216 |
|
217 |
## Training procedure
|
218 |
|