Hugging Face's logo

EricB
/
Phi-3.5-mini-instruct-UQFF

phi3
uqff
mistral.rs
custom_code
Model card Files
xet
Community
EricB HF Staff commited on
Commit
74019d6
·
verified ·
1 Parent(s): 947844a

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +5 -1
README.md CHANGED
@@ -21,5 +21,9 @@ Run with [mistral.rs](https://github.com/EricLBuehler/mistral.rs). Documentation
21
 
22
  |Name|Quantization type(s)|Example|
23
  |--|--|--|
24
- |phi3.5-mini-instruct-q4k.uqff|Q4K|`./mistralrs-server -i plain -m microsoft/Phi-3.5-mini-instruct --from-uqff EricB/Phi-3.5-mini-instruct-UQFF/phi3.5-mini-instruct-q4k.uqff`|
25
  |phi3.5-mini-instruct-hqq4.uqff|HQQ4|`./mistralrs-server -i plain -m microsoft/Phi-3.5-mini-instruct --from-uqff EricB/Phi-3.5-mini-instruct-UQFF/phi3.5-mini-instruct-hqq4.uqff`|
 
 
 
 
 
 
21
 
22
  |Name|Quantization type(s)|Example|
23
  |--|--|--|
 
24
  |phi3.5-mini-instruct-hqq4.uqff|HQQ4|`./mistralrs-server -i plain -m microsoft/Phi-3.5-mini-instruct --from-uqff EricB/Phi-3.5-mini-instruct-UQFF/phi3.5-mini-instruct-hqq4.uqff`|
25
+ |phi3.5-mini-instruct-hqq8.uqff|HQQ8|`./mistralrs-server -i plain -m microsoft/Phi-3.5-mini-instruct --from-uqff EricB/Phi-3.5-mini-instruct-UQFF/phi3.5-mini-instruct-hqq8.uqff`|
26
+ |phi3.5-mini-instruct-q4k.uqff|Q4K|`./mistralrs-server -i plain -m microsoft/Phi-3.5-mini-instruct --from-uqff EricB/Phi-3.5-mini-instruct-UQFF/phi3.5-mini-instruct-q4k.uqff`|
27
+ |phi3.5-mini-instruct-q5k.uqff|Q5K|`./mistralrs-server -i plain -m microsoft/Phi-3.5-mini-instruct --from-uqff EricB/Phi-3.5-mini-instruct-UQFF/phi3.5-mini-instruct-q5k.uqff`|
28
+ |phi3.5-mini-instruct-q6k.uqff|Q6K|`./mistralrs-server -i plain -m microsoft/Phi-3.5-mini-instruct --from-uqff EricB/Phi-3.5-mini-instruct-UQFF/phi3.5-mini-instruct-q6k.uqff`|
29
+ |phi3.5-mini-instruct-q8_0.uqff|Q8_0|`./mistralrs-server -i plain -m microsoft/Phi-3.5-mini-instruct --from-uqff EricB/Phi-3.5-mini-instruct-UQFF/phi3.5-mini-instruct-q8_0.uqff`|