sam-mosaic
commited on
Commit
•
fc67f07
1
Parent(s):
848c6bf
Update README.md
Browse files
README.md
CHANGED
@@ -24,7 +24,7 @@ tags:
|
|
24 |
inference: false
|
25 |
---
|
26 |
|
27 |
-
# MPT-
|
28 |
|
29 |
MPT-7B-8k-Chat is a chatbot-like model for dialogue generation.
|
30 |
It was built by finetuning [MPT-7B-8k](https://huggingface.co/mosaicml/mpt-7b-8k) on the [ShareGPT-Vicuna](https://huggingface.co/datasets/anon8231489123/ShareGPT_Vicuna_unfiltered), [Camel-AI](https://huggingface.co/camel-ai),
|
@@ -166,8 +166,7 @@ The model was trained on the following data mix:
|
|
166 |
|
167 |
### Training Configuration
|
168 |
|
169 |
-
|
170 |
-
This model was trained on **NUMBER** H100s for about **NUMBER** hours using the [MosaicML Platform](https://www.mosaicml.com/platform).
|
171 |
The model was trained with sharded data parallelism using [FSDP](https://pytorch.org/docs/stable/fsdp.html) and used the AdamW optimizer.
|
172 |
|
173 |
## Limitations and Biases
|
|
|
24 |
inference: false
|
25 |
---
|
26 |
|
27 |
+
# MPT-7B-Chat
|
28 |
|
29 |
MPT-7B-8k-Chat is a chatbot-like model for dialogue generation.
|
30 |
It was built by finetuning [MPT-7B-8k](https://huggingface.co/mosaicml/mpt-7b-8k) on the [ShareGPT-Vicuna](https://huggingface.co/datasets/anon8231489123/ShareGPT_Vicuna_unfiltered), [Camel-AI](https://huggingface.co/camel-ai),
|
|
|
166 |
|
167 |
### Training Configuration
|
168 |
|
169 |
+
This model was trained on 192 H100s for about 48 minutes using the [MosaicML Platform](https://www.mosaicml.com/platform).
|
|
|
170 |
The model was trained with sharded data parallelism using [FSDP](https://pytorch.org/docs/stable/fsdp.html) and used the AdamW optimizer.
|
171 |
|
172 |
## Limitations and Biases
|