Commit
·
077d34a
1
Parent(s):
7c507c0
Update README.md
Browse files
README.md
CHANGED
@@ -10,7 +10,8 @@ language:
|
|
10 |
inference: false
|
11 |
---
|
12 |
# Model Card for Mobius-12B-base-m1
|
13 |
-
The Mobius-12B-base-m1 Large Language Model (LLM) is a pretrained model based on RWKV v5 arch.
|
|
|
14 |
|
15 |
|
16 |
## Warning
|
@@ -51,6 +52,7 @@ print(tokenizer.decode(output[0].tolist(), skip_special_tokens=True))
|
|
51 |
## Limitations
|
52 |
|
53 |
The Mobius base m1 is the base model can be easily fine-tuned to achieve compelling performance.
|
|
|
54 |
|
55 |
### Benchmark
|
56 |
|
|
|
10 |
inference: false
|
11 |
---
|
12 |
# Model Card for Mobius-12B-base-m1
|
13 |
+
The Mobius-12B-base-m1 Large Language Model (LLM) is a pretrained model based on RWKV v5 arch.
|
14 |
+
We utilized 0.01 billion tokens to conduct post-training on this model for alignment benchmarks, excluding the utilization of SFT and DPO. The process took approximately 10 hours, employing 4 * a800.
|
15 |
|
16 |
|
17 |
## Warning
|
|
|
52 |
## Limitations
|
53 |
|
54 |
The Mobius base m1 is the base model can be easily fine-tuned to achieve compelling performance.
|
55 |
+
if you wanna better benchmark results use [DPO and SFT](https://github.com/BBuf/trl/pull/1) ,details in [readme](https://github.com/BBuf/trl/pull/1/files)
|
56 |
|
57 |
### Benchmark
|
58 |
|