Hugging Face's logo

MaziyarPanahi
/
Qwen2-1.5B-Instruct-GGUF

Text Generation
Transformers
GGUF
mistral
quantized
2-bit
3-bit
4-bit precision
5-bit
6-bit
8-bit precision
GGUF
llama-3
llama
imatrix
conversational
Model card Files Community
1
MaziyarPanahi commited on
Commit
8ff2a6a
•
1 Parent(s): d39ac8f

Upload folder using huggingface_hub (#1)

Browse files

- 1a00199528bcaa518112b9fb3713fe29f351e66c27af5d703d97361aceb990b5 (064f26d483a7d11224b6f139ba281d5e477a2c3d)
- d7b33f3b3a4c5822960c80b8e0aa102d5193b69d91f715218142b47f46dadcdf (9f0d8c19813cc80dec365828d7502058ff63fc3d)
- 1f11870748daf2ee4869f68ff8a4fc66ca8287bae85a773bddeae7ec64787d3a (19ad0e2cc0b007e2520a1dfd0c724793f91903e5)
- b50ce61c3ea9bccc0966f9f2f6728c83fadb808784b4f3559e4facf4b26ab0ed (0521d7ee8665c68cc8ef46cacf6eaa18e1f3a54a)
- 266263bc39e40ca9fc5d16d03dba0b74ebfef5fb8ffae5b4e94b320c446ac6a3 (2200f06989f2dc97255b2f82198fbc9b71ac8bc9)
- dae1d38225723125a6bf56fa94c590519406b1be9595eb89fb7141ab2e8c51b2 (fce14f7e4e0bd9bfd872bb28e3a5de2a222c5edf)
- 03e13749df7d6a028b16240dab2f1b73a99471386fbc528080769d17e29653da (369f2b4dc63bfd2470b875981b2e19894aea4e5a)
- 4f3d187783fc00913ea5bfadbd423ba649df4da7ad9ff024f16928e1ea2daaff (0378a199b81cb03ca35a554cceedad3940153c89)
- f53243bbbbda21fe7d5d3659a0629253e61c9a71dea47023b7b2181d4c66dc90 (6751034a0d6c650119f1db82206713319f4fd04a)
- da6cb90864a2fda86850409cacb7255c07acf03d6c4537f2abaa39b057cd74f3 (56c80efed9fb96dbb9c0a5e6ef41f1ecc9106998)

Files changed (18) hide show
  1. .gitattributes +16 -0
  2. Qwen2-1.5B-Instruct.IQ1_M.gguf +3 -0
  3. Qwen2-1.5B-Instruct.IQ1_S.gguf +3 -0
  4. Qwen2-1.5B-Instruct.IQ2_XS.gguf +3 -0
  5. Qwen2-1.5B-Instruct.IQ3_XS.gguf +3 -0
  6. Qwen2-1.5B-Instruct.IQ4_XS.gguf +3 -0
  7. Qwen2-1.5B-Instruct.Q2_K.gguf +3 -0
  8. Qwen2-1.5B-Instruct.Q3_K_L.gguf +3 -0
  9. Qwen2-1.5B-Instruct.Q3_K_M.gguf +3 -0
  10. Qwen2-1.5B-Instruct.Q3_K_S.gguf +3 -0
  11. Qwen2-1.5B-Instruct.Q4_K_M.gguf +3 -0
  12. Qwen2-1.5B-Instruct.Q4_K_S.gguf +3 -0
  13. Qwen2-1.5B-Instruct.Q5_K_M.gguf +3 -0
  14. Qwen2-1.5B-Instruct.Q5_K_S.gguf +3 -0
  15. Qwen2-1.5B-Instruct.Q6_K.gguf +3 -0
  16. Qwen2-1.5B-Instruct.Q8_0.gguf +3 -0
  17. Qwen2-1.5B-Instruct.fp16.gguf +3 -0
  18. README.md +48 -0
.gitattributes CHANGED
@@ -33,3 +33,19 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ Qwen2-1.5B-Instruct.IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text
37
+ Qwen2-1.5B-Instruct.IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text
38
+ Qwen2-1.5B-Instruct.IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text
39
+ Qwen2-1.5B-Instruct.IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
40
+ Qwen2-1.5B-Instruct.IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
41
+ Qwen2-1.5B-Instruct.Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
42
+ Qwen2-1.5B-Instruct.Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
43
+ Qwen2-1.5B-Instruct.Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
44
+ Qwen2-1.5B-Instruct.Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
45
+ Qwen2-1.5B-Instruct.Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
46
+ Qwen2-1.5B-Instruct.Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
47
+ Qwen2-1.5B-Instruct.Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
48
+ Qwen2-1.5B-Instruct.Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
49
+ Qwen2-1.5B-Instruct.Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
50
+ Qwen2-1.5B-Instruct.fp16.gguf filter=lfs diff=lfs merge=lfs -text
51
+ Qwen2-1.5B-Instruct.Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
Qwen2-1.5B-Instruct.IQ1_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:27224aae192d19dc7986fc394b5bd25f5ea71cb572b7207ce478c32e54808b01
3
+ size 464458560
Qwen2-1.5B-Instruct.IQ1_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c37a3e4c8d255a01b3a14147eddb443ac0088cf74bf27fb47c33ebcf6022c855
3
+ size 436524864
Qwen2-1.5B-Instruct.IQ2_XS.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bc1147c719e6db32a245997262cc6b2b1c458b77c199322728851fe649f2c8ba
3
+ size 550324032
Qwen2-1.5B-Instruct.IQ3_XS.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2a0edd87ee3def35ec883bd58d10245b196031de5d4ceed962ba9af1490f8f63
3
+ size 731696448
Qwen2-1.5B-Instruct.IQ4_XS.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:99cacf0d9ad427fcf4665bad0d4457a9284e295d91abc6e5dd1eb3400bc3fbf2
3
+ size 895728960
Qwen2-1.5B-Instruct.Q2_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9a3eb79f27019bce952eb2dd37b9cdd27ed5926ca561bf646f0c5ed1f9edc5d9
3
+ size 676302144
Qwen2-1.5B-Instruct.Q3_K_L.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:79b443d96abb17023b5639fd07296a42beed1e11acb5236327878acb477994f7
3
+ size 880160064
Qwen2-1.5B-Instruct.Q3_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:60d68aa2583aff811457ef01fe765c8b144a973af16e409b41a66294daf6ab8b
3
+ size 824175936
Qwen2-1.5B-Instruct.Q3_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2939c6adcfc28ff949e0a93526b2f8845abcf29631bfe1e5eafb81e97efa67a4
3
+ size 760941888
Qwen2-1.5B-Instruct.Q4_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5788f185a8c051df30193a157c6b18439d97d8584633eb87ffc086c53d3e465b
3
+ size 986045760
Qwen2-1.5B-Instruct.Q4_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:68dd25b418640b7b219fbdec41e6f22958177f455145b3c16b03f2c0a7be30d6
3
+ size 940309824
Qwen2-1.5B-Instruct.Q5_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:63007bde10afc7174f3606c05ce69213b843b7fd06e873f2ffe29ab93d824f2f
3
+ size 1125047616
Qwen2-1.5B-Instruct.Q5_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:602d6e35d89c895cd086b04cb7edb836e1974e863cc392041725067e54d61759
3
+ size 1098726720
Qwen2-1.5B-Instruct.Q6_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:58148c0e3025b575e546f9b58d1bd0e451c5ae9533c1ff15a94145061cd02538
3
+ size 1272737088
Qwen2-1.5B-Instruct.Q8_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b381ed02d573baa98aed3ffcf1e14cede136d19cdbbcfafb58b9225dc8307533
3
+ size 1646570304
Qwen2-1.5B-Instruct.fp16.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0fee06aea3b9278f0f170291208f56d0647450a5763d8ca1b4d9cc25610fa038
3
+ size 3093666432
README.md ADDED
@@ -0,0 +1,48 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ tags:
3
+ - quantized
4
+ - 2-bit
5
+ - 3-bit
6
+ - 4-bit
7
+ - 5-bit
8
+ - 6-bit
9
+ - 8-bit
10
+ - GGUF
11
+ - text-generation
12
+ - llama-3
13
+ - llama
14
+ - text-generation
15
+ model_name: Qwen2-1.5B-Instruct-GGUF
16
+ base_model: Qwen/Qwen2-1.5B-Instruct
17
+ inference: false
18
+ model_creator: Qwen
19
+ pipeline_tag: text-generation
20
+ quantized_by: MaziyarPanahi
21
+ ---
22
+ # [MaziyarPanahi/Qwen2-1.5B-Instruct-GGUF](https://huggingface.co/MaziyarPanahi/Qwen2-1.5B-Instruct-GGUF)
23
+ - Model creator: [Qwen](https://huggingface.co/Qwen)
24
+ - Original model: [Qwen/Qwen2-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2-1.5B-Instruct)
25
+
26
+ ## Description
27
+ [MaziyarPanahi/Qwen2-1.5B-Instruct-GGUF](https://huggingface.co/MaziyarPanahi/Qwen2-1.5B-Instruct-GGUF) contains GGUF format model files for [Qwen/Qwen2-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2-1.5B-Instruct).
28
+
29
+ ### About GGUF
30
+
31
+ GGUF is a new format introduced by the llama.cpp team on August 21st 2023. It is a replacement for GGML, which is no longer supported by llama.cpp.
32
+
33
+ Here is an incomplete list of clients and libraries that are known to support GGUF:
34
+
35
+ * [llama.cpp](https://github.com/ggerganov/llama.cpp). The source project for GGUF. Offers a CLI and a server option.
36
+ * [llama-cpp-python](https://github.com/abetlen/llama-cpp-python), a Python library with GPU accel, LangChain support, and OpenAI-compatible API server.
37
+ * [LM Studio](https://lmstudio.ai/), an easy-to-use and powerful local GUI for Windows and macOS (Silicon), with GPU acceleration. Linux available, in beta as of 27/11/2023.
38
+ * [text-generation-webui](https://github.com/oobabooga/text-generation-webui), the most widely used web UI, with many features and powerful extensions. Supports GPU acceleration.
39
+ * [KoboldCpp](https://github.com/LostRuins/koboldcpp), a fully featured web UI, with GPU accel across all platforms and GPU architectures. Especially good for story telling.
40
+ * [GPT4All](https://gpt4all.io/index.html), a free and open source local running GUI, supporting Windows, Linux and macOS with full GPU accel.
41
+ * [LoLLMS Web UI](https://github.com/ParisNeo/lollms-webui), a great web UI with many interesting and unique features, including a full model library for easy model selection.
42
+ * [Faraday.dev](https://faraday.dev/), an attractive and easy to use character-based chat GUI for Windows and macOS (both Silicon and Intel), with GPU acceleration.
43
+ * [candle](https://github.com/huggingface/candle), a Rust ML framework with a focus on performance, including GPU support, and ease of use.
44
+ * [ctransformers](https://github.com/marella/ctransformers), a Python library with GPU accel, LangChain support, and OpenAI-compatible AI server. Note, as of time of writing (November 27th 2023), ctransformers has not been updated in a long time and does not support many recent models.
45
+
46
+ ## Special thanks
47
+
48
+ 🙏 Special thanks to [Georgi Gerganov](https://github.com/ggerganov) and the whole team working on [llama.cpp](https://github.com/ggerganov/llama.cpp/) for making all of this possible.