ZeroWw commited on
Commit
2d84b92
·
verified ·
1 Parent(s): 5e9a572

Upload folder using huggingface_hub

Browse files
.gitattributes CHANGED
@@ -33,3 +33,9 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ granite-3.1-8b-instruct-abliterated.f16.gguf filter=lfs diff=lfs merge=lfs -text
37
+ granite-3.1-8b-instruct-abliterated.q5_k.gguf filter=lfs diff=lfs merge=lfs -text
38
+ granite-3.1-8b-instruct-abliterated.q6_k.gguf filter=lfs diff=lfs merge=lfs -text
39
+ granite-3.1-8b-instruct-abliterated.q8_0.gguf filter=lfs diff=lfs merge=lfs -text
40
+ granite-3.1-8b-instruct-abliterated.q8_p.gguf filter=lfs diff=lfs merge=lfs -text
41
+ granite-3.1-8b-instruct-abliterated.q8q4.gguf filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,17 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+
2
+ ---
3
+ license: mit
4
+ language:
5
+ - en
6
+ pipeline_tag: text-generation
7
+ ---
8
+
9
+ My own (ZeroWw) quantizations.
10
+ output and embed tensors quantized to f16.
11
+ all other tensors quantized to q5_k or q6_k.
12
+
13
+ Result:
14
+ both f16.q6 and f16.q5 are smaller than q8_0 standard quantization
15
+ and they perform as well as the pure f16.
16
+
17
+ Updated on: Sat Jan 11, 00:38:07
granite-3.1-8b-instruct-abliterated.f16.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a10f4aced2b512224f071897304484b952b224c3ff579d43e0b4130ced2b5359
3
+ size 16344105696
granite-3.1-8b-instruct-abliterated.q5_k.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3345e9f82f31feccb7aa307c1af33c886674ff1924ffc7125e655e09b87e0ec2
3
+ size 6034965216
granite-3.1-8b-instruct-abliterated.q6_k.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:63b3ac2f4199c6cffda823fc9fbbd1e77069a73e3f8f814495b8ce876ce38316
3
+ size 6942966496
granite-3.1-8b-instruct-abliterated.q8_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fcb46da1b94151dca4a5189d41ef52a3d2a643c077d076e1074a9c312c5d5715
3
+ size 8873001696
granite-3.1-8b-instruct-abliterated.q8_p.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2fb2e4e56844796dd3da972fbd8f84143922ba4ac36f6378a93081af1a731279
3
+ size 8684246496
granite-3.1-8b-instruct-abliterated.q8q4.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8e94ad5ffaaaa501bbb3dc670b820bde99a41a7b743e5bdfd16f76e566294ca1
3
+ size 4991620576
granite-3.1-8b-instruct-abliterated/README.md ADDED
@@ -0,0 +1,26 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ pipeline_tag: text-generation
3
+ inference: false
4
+ license: apache-2.0
5
+ library_name: transformers
6
+ tags:
7
+ - language
8
+ - granite-3.1
9
+ - abliterated
10
+ - uncensored
11
+ base_model:
12
+ - ibm-granite/granite-3.1-8b-instruct
13
+ ---
14
+
15
+ # huihui-ai/granite-3.1-8b-instruct-abliterated
16
+
17
+
18
+ This is an uncensored version of [ibm-granite/granite-3.1-8b-instruct](https://huggingface.co/ibm-granite/granite-3.1-8b-instruct) created with abliteration (see [remove-refusals-with-transformers](https://github.com/Sumandora/remove-refusals-with-transformers) to know more about it).
19
+ This is a crude, proof-of-concept implementation to remove refusals from an LLM model without using TransformerLens.
20
+
21
+ ## Use with ollama
22
+
23
+ You can use [huihui_ai/granite3.1-dense-abliterated](https://ollama.com/huihui_ai/granite3.1-dense-abliterated) directly,
24
+ ```
25
+ ollama run huihui_ai/granite3.1-dense-abliterated
26
+ ```