bartowski commited on
Commit
e118667
·
verified ·
1 Parent(s): fc3e665

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +7 -24
README.md CHANGED
@@ -1,45 +1,28 @@
1
  ---
2
  quantized_by: bartowski
3
  pipeline_tag: text-generation
4
- license_link: https://huggingface.co/microsoft/phi-4/resolve/main/LICENSE
5
- inference:
6
- parameters:
7
- temperature: 0
8
- widget:
9
- - messages:
10
- - role: user
11
- content: How should I explain the Internet?
12
- tags:
13
- - phi
14
- - nlp
15
- - math
16
- - code
17
- - chat
18
- - conversational
19
- language:
20
- - en
21
- base_model: microsoft/phi-4
22
- license: mit
23
  ---
24
 
25
  ## Llamacpp imatrix Quantizations of phi-4
26
 
27
- Using <a href="https://github.com/ggerganov/llama.cpp/">llama.cpp</a> release <a href="https://github.com/ggerganov/llama.cpp/releases/tag/b4418">b4418</a> for quantization.
28
 
29
  Original model: https://huggingface.co/microsoft/phi-4
30
 
31
  All quants made using imatrix option with dataset from [here](https://gist.github.com/bartowski1182/eb213dccb3571f863da82e99418f81e8)
32
 
33
- Run them in [LM Studio](https://lmstudio.ai/) with version 0.3.6
34
-
35
- Ollama doesn't yet support this (as of January 8 2025)
36
 
37
  ## Prompt format
38
 
39
  ```
40
- <|im_start|>system<|im_sep|>{system_prompt}<|im_end|><|im_start|>user<|im_sep|>{prompt}<|im_end|><|im_start|>assistant<|im_sep|><|im_end|>
41
  ```
42
 
 
 
 
 
43
  ## Download a file (not the whole branch) from below:
44
 
45
  | Filename | Quant type | File Size | Split | Description |
 
1
  ---
2
  quantized_by: bartowski
3
  pipeline_tag: text-generation
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
4
  ---
5
 
6
  ## Llamacpp imatrix Quantizations of phi-4
7
 
8
+ Using <a href="https://github.com/ggerganov/llama.cpp/">llama.cpp</a> release <a href="https://github.com/ggerganov/llama.cpp/releases/tag/b4456">b4456</a> for quantization.
9
 
10
  Original model: https://huggingface.co/microsoft/phi-4
11
 
12
  All quants made using imatrix option with dataset from [here](https://gist.github.com/bartowski1182/eb213dccb3571f863da82e99418f81e8)
13
 
14
+ Run them in [LM Studio](https://lmstudio.ai/)
 
 
15
 
16
  ## Prompt format
17
 
18
  ```
19
+ <|im_start|>system<|im_sep|>{system_prompt}<|im_end|><|im_start|>user<|im_sep|>{prompt}<|im_end|><|im_start|>assistant<|im_sep|>
20
  ```
21
 
22
+ ## What's new:
23
+
24
+ Tokenizer fixes as recommended by the unsloth team
25
+
26
  ## Download a file (not the whole branch) from below:
27
 
28
  | Filename | Quant type | File Size | Split | Description |