Hugging Face's logo

tensorblock
/
Yi-Ko-34B-GGUF

Text Generation
Transformers
GGUF
PyTorch
English
Korean
Yi-Ko
01-ai
Yi
TensorBlock
GGUF
Model card Files Community
Yi-Ko-34B-GGUF / README.md
morriszms's picture
morriszms
Upload folder using huggingface_hub
75e1639 verified
preview code
|
raw
history blame contribute delete
4.76 kB
metadata
extra_gated_heading: Access beomi/Yi-Ko-34B on Hugging Face
extra_gated_button_content: Submit
extra_gated_fields:
  I agree to share my name, email address and username: checkbox
  I confirm that I understand this project is for research purposes only, and confirm that I agree to follow the LICENSE of this model: checkbox
language:
  - en
  - ko
pipeline_tag: text-generation
inference: false
tags:
  - pytorch
  - Yi-Ko
  - 01-ai
  - Yi
  - TensorBlock
  - GGUF
library_name: transformers
license: apache-2.0
base_model: beomi/Yi-Ko-34B
TensorBlock

Feedback and support: TensorBlock's Twitter/X, Telegram Group and Discord server

beomi/Yi-Ko-34B - GGUF

This repo contains GGUF format model files for beomi/Yi-Ko-34B.

The files were quantized using machines provided by TensorBlock, and they are compatible with llama.cpp as of commit b4242.

Prompt template

Model file specification

Filename Quant type File Size Description
Yi-Ko-34B-Q2_K.gguf Q2_K 12.945 GB smallest, significant quality loss - not recommended for most purposes
Yi-Ko-34B-Q3_K_S.gguf Q3_K_S 15.090 GB very small, high quality loss
Yi-Ko-34B-Q3_K_M.gguf Q3_K_M 16.785 GB very small, high quality loss
Yi-Ko-34B-Q3_K_L.gguf Q3_K_L 18.269 GB small, substantial quality loss
Yi-Ko-34B-Q4_0.gguf Q4_0 19.610 GB legacy; small, very high quality loss - prefer using Q3_K_M
Yi-Ko-34B-Q4_K_S.gguf Q4_K_S 19.742 GB small, greater quality loss
Yi-Ko-34B-Q4_K_M.gguf Q4_K_M 20.802 GB medium, balanced quality - recommended
Yi-Ko-34B-Q5_0.gguf Q5_0 23.864 GB legacy; medium, balanced quality - prefer using Q4_K_M
Yi-Ko-34B-Q5_K_S.gguf Q5_K_S 23.864 GB large, low quality loss - recommended
Yi-Ko-34B-Q5_K_M.gguf Q5_K_M 24.479 GB large, very low quality loss - recommended
Yi-Ko-34B-Q6_K.gguf Q6_K 28.384 GB very large, extremely low quality loss
Yi-Ko-34B-Q8_0.gguf Q8_0 36.763 GB very large, extremely low quality loss - not recommended

Downloading instruction

Command line

Firstly, install Huggingface Client

pip install -U "huggingface_hub[cli]"

Then, downoad the individual model file the a local directory

huggingface-cli download tensorblock/Yi-Ko-34B-GGUF --include "Yi-Ko-34B-Q2_K.gguf" --local-dir MY_LOCAL_DIR

If you wanna download multiple model files with a pattern (e.g., *Q4_K*gguf), you can try:

huggingface-cli download tensorblock/Yi-Ko-34B-GGUF --local-dir MY_LOCAL_DIR --local-dir-use-symlinks False --include='*Q4_K*gguf'