Update README.md
Browse files
README.md
CHANGED
@@ -14,14 +14,14 @@ tags:
|
|
14 |
- fine-tuned
|
15 |
base_model: microsoft/phi-3-mini-4k-instruct
|
16 |
model-index:
|
17 |
-
- name: phi3
|
18 |
results: []
|
19 |
---
|
20 |
|
21 |
-
# phi3
|
22 |
-
 optimized for roleplaying conversations with a variety of character personas. The model speaks in a conversational format. Please
|
25 |
|
26 |
## Example Conversations
|
27 |
|
@@ -93,9 +93,9 @@ import torch
|
|
93 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
94 |
|
95 |
# Load in half precision for best balance of performance and quality
|
96 |
-
tokenizer = AutoTokenizer.from_pretrained("
|
97 |
model = AutoModelForCausalLM.from_pretrained(
|
98 |
-
"
|
99 |
torch_dtype=torch.float16,
|
100 |
device_map="auto"
|
101 |
)
|
@@ -113,9 +113,9 @@ quantization_config = BitsAndBytesConfig(
|
|
113 |
)
|
114 |
|
115 |
# Load in 8-bit
|
116 |
-
tokenizer = AutoTokenizer.from_pretrained("
|
117 |
model = AutoModelForCausalLM.from_pretrained(
|
118 |
-
"
|
119 |
quantization_config=quantization_config,
|
120 |
device_map="auto"
|
121 |
)
|
@@ -133,9 +133,9 @@ quantization_config = BitsAndBytesConfig(
|
|
133 |
)
|
134 |
|
135 |
# Load in 4-bit
|
136 |
-
tokenizer = AutoTokenizer.from_pretrained("
|
137 |
model = AutoModelForCausalLM.from_pretrained(
|
138 |
-
"
|
139 |
quantization_config=quantization_config,
|
140 |
device_map="auto"
|
141 |
)
|
@@ -144,7 +144,7 @@ model = AutoModelForCausalLM.from_pretrained(
|
|
144 |
**For CPU-only inference** (much slower but works on any system):
|
145 |
```python
|
146 |
model = AutoModelForCausalLM.from_pretrained(
|
147 |
-
"
|
148 |
device_map="cpu"
|
149 |
)
|
150 |
```
|
@@ -157,7 +157,7 @@ Note: Lower precision (8-bit and 4-bit) may result in slightly reduced output qu
|
|
157 |
The model has been optimized to maintain persona consistency while capable of adopting different characters. It excels at creative, character-driven conversations and exhibits a high degree of adaptability to different personality traits provided in the system prompt.
|
158 |
|
159 |
### Training Data
|
160 |
-
We are unable to open source the dataset at this time, due to its use for proprietary internal
|
161 |
|
162 |
- **Dataset Size**: ~13k high-quality examples (curated from 50k initial conversations)
|
163 |
- **Data Format**: JSONL with each entry containing a messages array with system, user, and assistant roles
|
@@ -168,9 +168,9 @@ We are unable to open source the dataset at this time, due to its use for propri
|
|
168 |
|
169 |
Training metrics show consistent improvement throughout the training process:
|
170 |
|
171 |
-

|
205 |
model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype=torch.float16, device_map="auto")
|
206 |
|
@@ -241,7 +241,7 @@ import torch
|
|
241 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
242 |
|
243 |
class CharacterChat:
|
244 |
-
def __init__(self, model_path="
|
245 |
print(f"Loading model from {model_path}...")
|
246 |
self.tokenizer = AutoTokenizer.from_pretrained(model_path)
|
247 |
self.model = AutoModelForCausalLM.from_pretrained(
|
|
|
14 |
- fine-tuned
|
15 |
base_model: microsoft/phi-3-mini-4k-instruct
|
16 |
model-index:
|
17 |
+
- name: luvai-phi3
|
18 |
results: []
|
19 |
---
|
20 |
|
21 |
+
# luvai-phi3
|
22 |
+

|
23 |
|
24 |
+
This model is a fine-tuned version of [microsoft/phi-3-mini-4k-instruct](https://huggingface.co/microsoft/phi-3-mini-4k-instruct) optimized for roleplaying conversations with a variety of character personas. The model speaks in a conversational format. Please note, prompt template guidelines are extremely important in getting usable output.
|
25 |
|
26 |
## Example Conversations
|
27 |
|
|
|
93 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
94 |
|
95 |
# Load in half precision for best balance of performance and quality
|
96 |
+
tokenizer = AutoTokenizer.from_pretrained("luvGPT/luvai-phi3")
|
97 |
model = AutoModelForCausalLM.from_pretrained(
|
98 |
+
"luvGPT/luvai-phi3",
|
99 |
torch_dtype=torch.float16,
|
100 |
device_map="auto"
|
101 |
)
|
|
|
113 |
)
|
114 |
|
115 |
# Load in 8-bit
|
116 |
+
tokenizer = AutoTokenizer.from_pretrained("luvGPT/luvai-phi3")
|
117 |
model = AutoModelForCausalLM.from_pretrained(
|
118 |
+
"luvGPT/luvai-phi3",
|
119 |
quantization_config=quantization_config,
|
120 |
device_map="auto"
|
121 |
)
|
|
|
133 |
)
|
134 |
|
135 |
# Load in 4-bit
|
136 |
+
tokenizer = AutoTokenizer.from_pretrained("luvGPT/luvai-phi3")
|
137 |
model = AutoModelForCausalLM.from_pretrained(
|
138 |
+
"luvGPT/luvai-phi3",
|
139 |
quantization_config=quantization_config,
|
140 |
device_map="auto"
|
141 |
)
|
|
|
144 |
**For CPU-only inference** (much slower but works on any system):
|
145 |
```python
|
146 |
model = AutoModelForCausalLM.from_pretrained(
|
147 |
+
"luvGPT/luvai-phi3",
|
148 |
device_map="cpu"
|
149 |
)
|
150 |
```
|
|
|
157 |
The model has been optimized to maintain persona consistency while capable of adopting different characters. It excels at creative, character-driven conversations and exhibits a high degree of adaptability to different personality traits provided in the system prompt.
|
158 |
|
159 |
### Training Data
|
160 |
+
We are unable to open source the dataset at this time, due to its use for proprietary internal luvGPT development. Initial conversations were generated by open source large language models given specific generation instructions and curated by a judge model.
|
161 |
|
162 |
- **Dataset Size**: ~13k high-quality examples (curated from 50k initial conversations)
|
163 |
- **Data Format**: JSONL with each entry containing a messages array with system, user, and assistant roles
|
|
|
168 |
|
169 |
Training metrics show consistent improvement throughout the training process:
|
170 |
|
171 |
+

|
172 |
|
173 |
+

|
174 |
|
175 |
- **Token Accuracy**: Improved from ~0.48 to ~0.73
|
176 |
- **Training Loss**: Decreased from ~2.2 to ~1.05
|
|
|
200 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
201 |
|
202 |
# Load model and tokenizer
|
203 |
+
model_name = "luvGPT/luvai-phi3"
|
204 |
tokenizer = AutoTokenizer.from_pretrained(model_name)
|
205 |
model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype=torch.float16, device_map="auto")
|
206 |
|
|
|
241 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
242 |
|
243 |
class CharacterChat:
|
244 |
+
def __init__(self, model_path="luvGPT/luvai-phi3", persona=None):
|
245 |
print(f"Loading model from {model_path}...")
|
246 |
self.tokenizer = AutoTokenizer.from_pretrained(model_path)
|
247 |
self.model = AutoModelForCausalLM.from_pretrained(
|