Update README.md
Browse files
README.md
CHANGED
@@ -1,36 +1,38 @@
|
|
1 |
-
---
|
2 |
-
language:
|
3 |
-
- en
|
4 |
-
license: apache-2.0
|
5 |
-
base_model: unsloth/llama-3-8b-Instruct-bnb-4bit
|
6 |
-
---
|
7 |
-
|
8 |
-
LoRA for LLama-3-8b-Instruct, trained using dataset based on [toxicqa](https://huggingface.co/datasets/NobodyExistsOnTheInternet/toxicqa) and [toxic-dpo-v0.2](https://huggingface.co/datasets/unalignment/toxic-dpo-v0.2). The model does not refuse to follow instructions, and may give provocative answers when asked about private views.
|
9 |
-
|
10 |
-
# Usage
|
11 |
-
|
12 |
-
Recommended prompt format: Alpaca
|
13 |
-
|
14 |
-
Repository contains peft and gguf versions.
|
15 |
-
Base model for peft version: [Meta-Llama-3-8B-Instruct](https://huggingface.co/NousResearch/Meta-Llama-3-8B-Instruct)
|
16 |
-
Base model for gguf version: [Meta-Llama-3-8B-Instruct-GGUF](https://huggingface.co/QuantFactory/Meta-Llama-3-8B-Instruct-GGUF)
|
17 |
-
|
18 |
-
Use [koboldcpp](https://github.com/LostRuins/koboldcpp) or [text-generation-webui](https://github.com/oobabooga/text-generation-webui) to run it.
|
19 |
-
|
20 |
-
# Training parameters
|
21 |
-
|
22 |
-
method: ORPO
|
23 |
-
learning_rate: 1e-5
|
24 |
-
train_batch_size: 4
|
25 |
-
gradient_accumulation_steps: 4
|
26 |
-
total_train_batch_size: 16
|
27 |
-
optimizer: paged_adamw_8bit
|
28 |
-
lr_scheduler_type: cosine
|
29 |
-
lr_scheduler_warmup_steps: 100
|
30 |
-
num_steps: 1200
|
31 |
-
|
32 |
-
# Usage permission
|
33 |
-
|
34 |
-
You may use the contents of the repository in any manner consistent with the license and applicable law.
|
35 |
-
|
36 |
-
|
|
|
|
|
|
1 |
+
---
|
2 |
+
language:
|
3 |
+
- en
|
4 |
+
license: apache-2.0
|
5 |
+
base_model: unsloth/llama-3-8b-Instruct-bnb-4bit
|
6 |
+
---
|
7 |
+
|
8 |
+
LoRA for LLama-3-8b-Instruct, trained using dataset based on [toxicqa](https://huggingface.co/datasets/NobodyExistsOnTheInternet/toxicqa) and [toxic-dpo-v0.2](https://huggingface.co/datasets/unalignment/toxic-dpo-v0.2). The model does not refuse to follow instructions, and may give provocative answers when asked about private views.
|
9 |
+
|
10 |
+
# Usage
|
11 |
+
|
12 |
+
Recommended prompt format: Alpaca
|
13 |
+
|
14 |
+
Repository contains peft and gguf versions.
|
15 |
+
Base model for peft version: [Meta-Llama-3-8B-Instruct](https://huggingface.co/NousResearch/Meta-Llama-3-8B-Instruct)
|
16 |
+
Base model for gguf version: [Meta-Llama-3-8B-Instruct-GGUF](https://huggingface.co/QuantFactory/Meta-Llama-3-8B-Instruct-GGUF)
|
17 |
+
|
18 |
+
Use [koboldcpp](https://github.com/LostRuins/koboldcpp) or [text-generation-webui](https://github.com/oobabooga/text-generation-webui) to run it.
|
19 |
+
|
20 |
+
# Training parameters
|
21 |
+
|
22 |
+
method: ORPO
|
23 |
+
learning_rate: 1e-5
|
24 |
+
train_batch_size: 4
|
25 |
+
gradient_accumulation_steps: 4
|
26 |
+
total_train_batch_size: 16
|
27 |
+
optimizer: paged_adamw_8bit
|
28 |
+
lr_scheduler_type: cosine
|
29 |
+
lr_scheduler_warmup_steps: 100
|
30 |
+
num_steps: 1200
|
31 |
+
|
32 |
+
# Usage permission
|
33 |
+
|
34 |
+
You may use the contents of the repository in any manner consistent with the license and applicable law.
|
35 |
+
You are solely responsible for downloading and using the contents of the repository.
|
36 |
+
Keep in mind that the content generated by the model does not refer in any way to the views of the author or those known to him.
|
37 |
+
|
38 |
+
[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
|