This repo contains quantized large language model(LLM) weight files in GGUF format for maywell/kiqu-70b. The IQ quantized model files are calibrated with 20k_random_data.txt

Quant Type Size BPW Perplexity
IQ1_S 14.5 GB 1.5625 โ€“ 1.69 16.5308 +/- 0.13137
IQ2_XXS 18.3 GB 2.0625 โ€“ 2.12 12.1174 +/- 0.09202
IQ2_XS 20.3 GB 2.3125 โ€“ 2.36 11.2679 +/- 0.08525
IQ3_XXS 27 GB 3.0625 โ€“ 3.13 10.0546 +/- 0.07674
Q2_K 25.5 GB 2.95 4.2965 +/- 0.02164
Q4_0 38.9 GB 4.51 3.7527 +/- 0.01835
Q5_K_M 48.8 GB 5.65 3.7237 +/- 0.01833

kiqu-70b (Arena Leaderboard)

kiqu-70B

kiqu-70b is a SFT+DPO trained model based on Miqu-70B-Alpaca-DPO using Korean datasets.

Since this model is finetune of miqu-1-70b using it on commercial purposes is at your own risk. โ€” leaked early version Mistral-Medium

๋ณธ ๋ชจ๋ธ kiqu-70b๋Š” Miqu-70B-Alpaca-DPO ๋ชจ๋ธ์„ ๊ธฐ๋ฐ˜์œผ๋กœ ํ•œ๊ตญ์–ด ๋ฐ์ดํ„ฐ์…‹์„ ์‚ฌ์šฉํ•˜์—ฌ SFT+DPO ํ›ˆ๋ จ์„ ์ง„ํ–‰ํ•˜์—ฌ ์ œ์ž‘๋˜์—ˆ์Šต๋‹ˆ๋‹ค.

๋ฒ ์ด์Šค ๋ชจ๋ธ์ธ miqu-1-70b ๋ชจ๋ธ์ด ๋ฏธ์ŠคํŠธ๋ž„-๋ฏธ๋””์›€์˜ ์ดˆ๊ธฐ ์œ ์ถœ ๋ฒ„์ „์ด๊ธฐ์— ์ƒ์—…์  ์‚ฌ์šฉ์— ๋Œ€ํ•œ risk๋Š” ๋ณธ์ธ์—๊ฒŒ ์žˆ์Šต๋‹ˆ๋‹ค.

Beside that this model follows cc-by-sa-4.0

๋ณธ ๋ชจ๋ธ ์ž์ฒด๋กœ์„œ๋Š” cc-by-sa-4.0์„ ๋”ฐ๋ฆ…๋‹ˆ๋‹ค.

Model Details

Base Model
miqu-1-70b (Early Mistral-Medium)

Instruction format

It follows Mistral format. Giving few-shots to model is highly recommended

๋ณธ ๋ชจ๋ธ์€ ๋ฏธ์ŠคํŠธ๋ž„ ํฌ๋งท์„ ๋”ฐ๋ฆ…๋‹ˆ๋‹ค. few-shot ์‚ฌ์šฉ์„ ์ ๊ทน ๊ถŒ์žฅํ•ฉ๋‹ˆ๋‹ค.

[INST] {instruction}
[/INST] {output}

Multi-shot

[INST] {instruction}
[/INST] {output}

[INST] {instruction}
[/INST] {output}

[INST] {instruction}
[/INST] {output}
.
.
.

Recommended Template - 1-shot with system prompt

๋„ˆ๋Š” kiqu-70B๋ผ๋Š” ํ•œ๊ตญ์–ด์— ํŠนํ™”๋œ ์–ธ์–ด๋ชจ๋ธ์ด์•ผ. ๊น”๋”ํ•˜๊ณ  ์ž์—ฐ์Šค๋Ÿฝ๊ฒŒ ๋Œ€๋‹ตํ•ด์ค˜!
[INST] ์•ˆ๋…•?
[/INST] ์•ˆ๋…•ํ•˜์„ธ์š”! ๋ฌด์—‡์„ ๋„์™€๋“œ๋ฆด๊นŒ์š”? ์งˆ๋ฌธ์ด๋‚˜ ๊ถ๊ธˆํ•œ ์ ์ด ์žˆ๋‹ค๋ฉด ์–ธ์ œ๋“ ์ง€ ๋ง์”€ํ•ด์ฃผ์„ธ์š”.

[INST] {instruction}
[/INST]

Trailing space after [/INST] can affect models performance in significant margin. So, when doing inference it is recommended to not include trailing space in chat template.

[/INST] ๋’ค์— ๋„์–ด์“ฐ๊ธฐ๋Š” ๋ชจ๋ธ ์„ฑ๋Šฅ์— ์œ ์˜๋ฏธํ•œ ์˜ํ–ฅ์„ ๋ฏธ์นฉ๋‹ˆ๋‹ค. ๋”ฐ๋ผ์„œ, ์ธํผ๋Ÿฐ์Šค(์ถ”๋ก )๊ณผ์ •์—์„œ๋Š” ์ฑ— ํ…œํ”Œ๋ฆฟ์— ๋„์–ด์“ฐ๊ธฐ๋ฅผ ์ œ์™ธํ•˜๋Š” ๊ฒƒ์„ ์ ๊ทน ๊ถŒ์žฅํ•ฉ๋‹ˆ๋‹ค.

Model Benchmark

TBD

Author's Message

This model's training got sponsered by no one but support from people around Earth.

Support Me

Discord Server

Contact Me on Discord - is.maywell

Follow me on twitter - https://twitter.com/stablefluffy

Downloads last month
57
GGUF
Model size
69B params
Architecture
llama

1-bit

2-bit

3-bit

4-bit

5-bit

Inference Examples
Inference API (serverless) has been turned off for this model.