Text Generation
PEFT
Safetensors
mistral
conversational
Eval Results
dfurman commited on
Commit
9f4ca4d
1 Parent(s): dde117d

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +19 -19
README.md CHANGED
@@ -19,23 +19,23 @@ base_model: mistralai/Mixtral-8x7B-v0.1
19
  </div>
20
 
21
 
22
- # Mistral-7B-Instruct-v0.2
23
 
24
- A pretrained generative language model with 7 billion parameters geared towards instruction-following capabilities.
25
 
26
  ## Model Details
27
 
28
- This model was built via parameter-efficient finetuning of the [mistralai/Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1) base model on the first 20k rows in each of the [jondurbin/airoboros-2.2.1](https://huggingface.co/datasets/jondurbin/airoboros-2.2.1), [Open-Orca/SlimOrca](https://huggingface.co/datasets/Open-Orca/SlimOrca), and [garage-bAInd/Open-Platypus](https://huggingface.co/datasets/garage-bAInd/Open-Platypus) datasets.
29
 
30
  - **Developed by:** Daniel Furman
31
  - **Model type:** Causal language model (clm)
32
  - **Language(s) (NLP):** English
33
  - **License:** Apache 2.0
34
- - **Finetuned from model:** [mistralai/Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1)
35
 
36
  ## Model Sources
37
 
38
- - **Repository:** [here](https://github.com/daniel-furman/sft-demos/blob/main/src/sft/mistral/sft_Mistral_7B_Instruct_v0_1_peft.ipynb)
39
 
40
  ## Evaluation Results
41
 
@@ -69,7 +69,7 @@ from transformers import (
69
  ```
70
 
71
  ```python
72
- peft_model_id = "dfurman/Mistral-7B-Instruct-v0.2"
73
  config = PeftConfig.from_pretrained(peft_model_id)
74
 
75
  tokenizer = AutoTokenizer.from_pretrained(
@@ -141,18 +141,18 @@ print(response)
141
  **Generation**:
142
 
143
  ```python
144
- """1. Combine the following ingredients in a cocktail shaker:
145
- 2 oz light rum (or white rum)
146
- 1 oz dark rum
147
- 0.5 oz orange curacao or triple sec
148
- 0.75 oz lime juice, freshly squeezed
149
- 0.5 tbsp simple syrup (optional; if you like your drinks sweet)
150
- Few drops of bitters (Angostura is traditional but any will do)
151
- Ice cubes to fill the shaker
152
-
153
- 2. Shake vigorously until well-chilled and combined.
154
- 3. Strain into an ice-filled glass.
155
- 4. Garnish with a slice of lime or an orange wedge, if desired."""
156
  ```
157
 
158
  </details>
@@ -166,7 +166,7 @@ Ice cubes to fill the shaker
166
 
167
  ## Training
168
 
169
- It took ~5 hours to train 3 epochs on 1x A100 (40 GB SXM).
170
 
171
  ### Prompt Format
172
 
 
19
  </div>
20
 
21
 
22
+ # Mixtral-8x7B-Instruct-v0.1
23
 
24
+ A pretrained generative language model with ~47 billion parameters geared towards instruction-following capabilities.
25
 
26
  ## Model Details
27
 
28
+ This model was built via parameter-efficient finetuning of the [mistralai/Mixtral-8x7B-v0.1](https://huggingface.co/mistralai/Mixtral-8x7B-v0.1) base model on the first 20k rows in each of the [jondurbin/airoboros-2.2.1](https://huggingface.co/datasets/jondurbin/airoboros-2.2.1), [Open-Orca/SlimOrca](https://huggingface.co/datasets/Open-Orca/SlimOrca), and [garage-bAInd/Open-Platypus](https://huggingface.co/datasets/garage-bAInd/Open-Platypus) datasets.
29
 
30
  - **Developed by:** Daniel Furman
31
  - **Model type:** Causal language model (clm)
32
  - **Language(s) (NLP):** English
33
  - **License:** Apache 2.0
34
+ - **Finetuned from model:** [mistralai/Mixtral-8x7B-v0.1](https://huggingface.co/mistralai/Mixtral-8x7B-v0.1)
35
 
36
  ## Model Sources
37
 
38
+ - **Repository:** [here](https://github.com/daniel-furman/sft-demos/blob/main/src/sft/mixtral/sft_Mixtral_8x7B_Instruct_v0_1_peft.py)
39
 
40
  ## Evaluation Results
41
 
 
69
  ```
70
 
71
  ```python
72
+ peft_model_id = "dfurman/Mixtral-8x7B-Instruct-v0.1"
73
  config = PeftConfig.from_pretrained(peft_model_id)
74
 
75
  tokenizer = AutoTokenizer.from_pretrained(
 
141
  **Generation**:
142
 
143
  ```python
144
+ """1.5 oz White Rum
145
+ 2 oz Dark Rum
146
+ 1 oz Orange Curacao
147
+ 0.5 oz Orgeat Syrup
148
+ 0.5 oz Simple Syrup
149
+ 0.75 oz Lime Juice
150
+
151
+ In a shaker filled with ice, combine the white rum, dark rum, orange curacao, orgeat syrup, simple syrup, and lime juice. Shake vigorously for 10-15 seconds.
152
+
153
+ Strain the mixture into a double old-fashioned glass filled with fresh ice. Garnish with a lime wedge and a sprig of mint.
154
+
155
+ Enjoy your delicious mai tai!"""
156
  ```
157
 
158
  </details>
 
166
 
167
  ## Training
168
 
169
+ It took ~24 hours to train 2 epochs on 4x A6000s.
170
 
171
  ### Prompt Format
172