Hugging Face's logo

tensorblock
/
dolphin-2.9.2-Phi-3-Medium-abliterated-GGUF

GGUF
English
TensorBlock
GGUF
Inference Endpoints
conversational
Model card Files Community
morriszms commited on
Commit
27d3f07
•
1 Parent(s): ff19946

Upload folder using huggingface_hub

Browse files
Files changed (14) hide show
  1. .gitattributes +12 -0
  2. README.md +85 -0
  3. dolphin-2.9.2-Phi-3-Medium-abliterated-Q2_K.gguf +3 -0
  4. dolphin-2.9.2-Phi-3-Medium-abliterated-Q3_K_L.gguf +3 -0
  5. dolphin-2.9.2-Phi-3-Medium-abliterated-Q3_K_M.gguf +3 -0
  6. dolphin-2.9.2-Phi-3-Medium-abliterated-Q3_K_S.gguf +3 -0
  7. dolphin-2.9.2-Phi-3-Medium-abliterated-Q4_0.gguf +3 -0
  8. dolphin-2.9.2-Phi-3-Medium-abliterated-Q4_K_M.gguf +3 -0
  9. dolphin-2.9.2-Phi-3-Medium-abliterated-Q4_K_S.gguf +3 -0
  10. dolphin-2.9.2-Phi-3-Medium-abliterated-Q5_0.gguf +3 -0
  11. dolphin-2.9.2-Phi-3-Medium-abliterated-Q5_K_M.gguf +3 -0
  12. dolphin-2.9.2-Phi-3-Medium-abliterated-Q5_K_S.gguf +3 -0
  13. dolphin-2.9.2-Phi-3-Medium-abliterated-Q6_K.gguf +3 -0
  14. dolphin-2.9.2-Phi-3-Medium-abliterated-Q8_0.gguf +3 -0
.gitattributes CHANGED
@@ -33,3 +33,15 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
 
 
 
 
 
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ dolphin-2.9.2-Phi-3-Medium-abliterated-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
37
+ dolphin-2.9.2-Phi-3-Medium-abliterated-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
38
+ dolphin-2.9.2-Phi-3-Medium-abliterated-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
39
+ dolphin-2.9.2-Phi-3-Medium-abliterated-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
40
+ dolphin-2.9.2-Phi-3-Medium-abliterated-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
41
+ dolphin-2.9.2-Phi-3-Medium-abliterated-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
42
+ dolphin-2.9.2-Phi-3-Medium-abliterated-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
43
+ dolphin-2.9.2-Phi-3-Medium-abliterated-Q5_0.gguf filter=lfs diff=lfs merge=lfs -text
44
+ dolphin-2.9.2-Phi-3-Medium-abliterated-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
45
+ dolphin-2.9.2-Phi-3-Medium-abliterated-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
46
+ dolphin-2.9.2-Phi-3-Medium-abliterated-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
47
+ dolphin-2.9.2-Phi-3-Medium-abliterated-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,85 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ language:
4
+ - en
5
+ base_model: cognitivecomputations/dolphin-2.9.2-Phi-3-Medium-abliterated
6
+ datasets:
7
+ - cognitivecomputations/Dolphin-2.9.2
8
+ - teknium/OpenHermes-2.5
9
+ - m-a-p/CodeFeedback-Filtered-Instruction
10
+ - cognitivecomputations/dolphin-coder
11
+ - cognitivecomputations/samantha-data
12
+ - microsoft/orca-math-word-problems-200k
13
+ - internlm/Agent-FLAN
14
+ - cognitivecomputations/SystemChat-2.0
15
+ tags:
16
+ - TensorBlock
17
+ - GGUF
18
+ ---
19
+
20
+ <div style="width: auto; margin-left: auto; margin-right: auto">
21
+ <img src="https://i.imgur.com/jC7kdl8.jpeg" alt="TensorBlock" style="width: 100%; min-width: 400px; display: block; margin: auto;">
22
+ </div>
23
+ <div style="display: flex; justify-content: space-between; width: 100%;">
24
+ <div style="display: flex; flex-direction: column; align-items: flex-start;">
25
+ <p style="margin-top: 0.5em; margin-bottom: 0em;">
26
+ Feedback and support: TensorBlock's <a href="https://x.com/tensorblock_aoi">Twitter/X</a>, <a href="https://t.me/TensorBlock">Telegram Group</a> and <a href="https://x.com/tensorblock_aoi">Discord server</a>
27
+ </p>
28
+ </div>
29
+ </div>
30
+
31
+ ## cognitivecomputations/dolphin-2.9.2-Phi-3-Medium-abliterated - GGUF
32
+
33
+ This repo contains GGUF format model files for [cognitivecomputations/dolphin-2.9.2-Phi-3-Medium-abliterated](https://huggingface.co/cognitivecomputations/dolphin-2.9.2-Phi-3-Medium-abliterated).
34
+
35
+ The files were quantized using machines provided by [TensorBlock](https://tensorblock.co/), and they are compatible with llama.cpp as of [commit b4011](https://github.com/ggerganov/llama.cpp/commit/a6744e43e80f4be6398fc7733a01642c846dce1d).
36
+
37
+ ## Prompt template
38
+
39
+ ```
40
+ <|im_start|>system
41
+ {system_prompt}<|im_end|>
42
+ <|im_start|>user
43
+ {prompt}<|im_end|>
44
+ <|im_start|>assistant
45
+ ```
46
+
47
+ ## Model file specification
48
+
49
+ | Filename | Quant type | File Size | Description |
50
+ | -------- | ---------- | --------- | ----------- |
51
+ | [dolphin-2.9.2-Phi-3-Medium-abliterated-Q2_K.gguf](https://huggingface.co/tensorblock/dolphin-2.9.2-Phi-3-Medium-abliterated-GGUF/tree/main/dolphin-2.9.2-Phi-3-Medium-abliterated-Q2_K.gguf) | Q2_K | 4.847 GB | smallest, significant quality loss - not recommended for most purposes |
52
+ | [dolphin-2.9.2-Phi-3-Medium-abliterated-Q3_K_S.gguf](https://huggingface.co/tensorblock/dolphin-2.9.2-Phi-3-Medium-abliterated-GGUF/tree/main/dolphin-2.9.2-Phi-3-Medium-abliterated-Q3_K_S.gguf) | Q3_K_S | 5.648 GB | very small, high quality loss |
53
+ | [dolphin-2.9.2-Phi-3-Medium-abliterated-Q3_K_M.gguf](https://huggingface.co/tensorblock/dolphin-2.9.2-Phi-3-Medium-abliterated-GGUF/tree/main/dolphin-2.9.2-Phi-3-Medium-abliterated-Q3_K_M.gguf) | Q3_K_M | 6.287 GB | very small, high quality loss |
54
+ | [dolphin-2.9.2-Phi-3-Medium-abliterated-Q3_K_L.gguf](https://huggingface.co/tensorblock/dolphin-2.9.2-Phi-3-Medium-abliterated-GGUF/tree/main/dolphin-2.9.2-Phi-3-Medium-abliterated-Q3_K_L.gguf) | Q3_K_L | 6.844 GB | small, substantial quality loss |
55
+ | [dolphin-2.9.2-Phi-3-Medium-abliterated-Q4_0.gguf](https://huggingface.co/tensorblock/dolphin-2.9.2-Phi-3-Medium-abliterated-GGUF/tree/main/dolphin-2.9.2-Phi-3-Medium-abliterated-Q4_0.gguf) | Q4_0 | 7.355 GB | legacy; small, very high quality loss - prefer using Q3_K_M |
56
+ | [dolphin-2.9.2-Phi-3-Medium-abliterated-Q4_K_S.gguf](https://huggingface.co/tensorblock/dolphin-2.9.2-Phi-3-Medium-abliterated-GGUF/tree/main/dolphin-2.9.2-Phi-3-Medium-abliterated-Q4_K_S.gguf) | Q4_K_S | 7.411 GB | small, greater quality loss |
57
+ | [dolphin-2.9.2-Phi-3-Medium-abliterated-Q4_K_M.gguf](https://huggingface.co/tensorblock/dolphin-2.9.2-Phi-3-Medium-abliterated-GGUF/tree/main/dolphin-2.9.2-Phi-3-Medium-abliterated-Q4_K_M.gguf) | Q4_K_M | 7.827 GB | medium, balanced quality - recommended |
58
+ | [dolphin-2.9.2-Phi-3-Medium-abliterated-Q5_0.gguf](https://huggingface.co/tensorblock/dolphin-2.9.2-Phi-3-Medium-abliterated-GGUF/tree/main/dolphin-2.9.2-Phi-3-Medium-abliterated-Q5_0.gguf) | Q5_0 | 8.961 GB | legacy; medium, balanced quality - prefer using Q4_K_M |
59
+ | [dolphin-2.9.2-Phi-3-Medium-abliterated-Q5_K_S.gguf](https://huggingface.co/tensorblock/dolphin-2.9.2-Phi-3-Medium-abliterated-GGUF/tree/main/dolphin-2.9.2-Phi-3-Medium-abliterated-Q5_K_S.gguf) | Q5_K_S | 8.961 GB | large, low quality loss - recommended |
60
+ | [dolphin-2.9.2-Phi-3-Medium-abliterated-Q5_K_M.gguf](https://huggingface.co/tensorblock/dolphin-2.9.2-Phi-3-Medium-abliterated-GGUF/tree/main/dolphin-2.9.2-Phi-3-Medium-abliterated-Q5_K_M.gguf) | Q5_K_M | 9.204 GB | large, very low quality loss - recommended |
61
+ | [dolphin-2.9.2-Phi-3-Medium-abliterated-Q6_K.gguf](https://huggingface.co/tensorblock/dolphin-2.9.2-Phi-3-Medium-abliterated-GGUF/tree/main/dolphin-2.9.2-Phi-3-Medium-abliterated-Q6_K.gguf) | Q6_K | 10.667 GB | very large, extremely low quality loss |
62
+ | [dolphin-2.9.2-Phi-3-Medium-abliterated-Q8_0.gguf](https://huggingface.co/tensorblock/dolphin-2.9.2-Phi-3-Medium-abliterated-GGUF/tree/main/dolphin-2.9.2-Phi-3-Medium-abliterated-Q8_0.gguf) | Q8_0 | 13.816 GB | very large, extremely low quality loss - not recommended |
63
+
64
+
65
+ ## Downloading instruction
66
+
67
+ ### Command line
68
+
69
+ Firstly, install Huggingface Client
70
+
71
+ ```shell
72
+ pip install -U "huggingface_hub[cli]"
73
+ ```
74
+
75
+ Then, downoad the individual model file the a local directory
76
+
77
+ ```shell
78
+ huggingface-cli download tensorblock/dolphin-2.9.2-Phi-3-Medium-abliterated-GGUF --include "dolphin-2.9.2-Phi-3-Medium-abliterated-Q2_K.gguf" --local-dir MY_LOCAL_DIR
79
+ ```
80
+
81
+ If you wanna download multiple model files with a pattern (e.g., `*Q4_K*gguf`), you can try:
82
+
83
+ ```shell
84
+ huggingface-cli download tensorblock/dolphin-2.9.2-Phi-3-Medium-abliterated-GGUF --local-dir MY_LOCAL_DIR --local-dir-use-symlinks False --include='*Q4_K*gguf'
85
+ ```
dolphin-2.9.2-Phi-3-Medium-abliterated-Q2_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:22fdbaf0cdcb3f10f26aaa12d6289f5c0fbb128f6147db2d501b1b9d27dc184f
3
+ size 5204447808
dolphin-2.9.2-Phi-3-Medium-abliterated-Q3_K_L.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f1ed79be4373491eb0d0a5ed270e6d8604b3086442250060fa06f854c7ba9835
3
+ size 7348993088
dolphin-2.9.2-Phi-3-Medium-abliterated-Q3_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0bce1b6ec7df0c0210101e4067581fca38069ba35191d5eb1b565150534657eb
3
+ size 6750977088
dolphin-2.9.2-Phi-3-Medium-abliterated-Q3_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4cfa7aab643ddaa48f81c7226a78fbe9cc668219a77e53746f701c059fe5547c
3
+ size 6064897088
dolphin-2.9.2-Phi-3-Medium-abliterated-Q4_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:42218638933c3e4572f166aaee60c13f9b2413eba5f26b4f5a9475ef734e2b46
3
+ size 7897132608
dolphin-2.9.2-Phi-3-Medium-abliterated-Q4_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1e0378c331cd36542966ad8bab407b8fb93068be7921a8f65d9c3b103ed6e717
3
+ size 8404012608
dolphin-2.9.2-Phi-3-Medium-abliterated-Q4_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1eab18c65cc9d31fe033140c3565c6119576f39db6238a43885a4a1e622323df
3
+ size 7957753408
dolphin-2.9.2-Phi-3-Medium-abliterated-Q5_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:90a555c0612968e619cf47d20f6b68e3e483ea7088782ff8318e22beb2e50420
3
+ size 9621589568
dolphin-2.9.2-Phi-3-Medium-abliterated-Q5_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7c581112bea15f92cfd64543bd3467593dad74a4c85d8b7374041fcb73c930c9
3
+ size 9882709568
dolphin-2.9.2-Phi-3-Medium-abliterated-Q5_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2f18f3597484d67ce9389f503161beee801fbb905a1eab299e4aac8a6f4d821d
3
+ size 9621589568
dolphin-2.9.2-Phi-3-Medium-abliterated-Q6_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1e8c725c399c62ed513132ee8ce90235e1e71f60020faaaba26d6e896c3449b5
3
+ size 11453825088
dolphin-2.9.2-Phi-3-Medium-abliterated-Q8_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:38949587ffde2d56f04009ca68d95d3910c15010b1077861d77d02e33daef756
3
+ size 14834719808