Llama-3-Taiwan-8B-Instruct - GPTQ

Model creator: Yen-Ting Lin
Original model: Llama-3-Taiwan-8B-Instruct

Description

This repo contains GPTQ model files for Llama-3-Taiwan-8B-Instruct.

GPTQ models for GPU inference
Yen-Ting Lin's original unquantized model

Quantization parameter

Bits : 4
Group Size : 128
Act Order : Yes
Damp % : 0.1
Seq Len : 2048
Size : 5.34 GB

It tooks about 50 mins to quantize on H100.

Downloads last month: 5

Safetensors

Model size

1.99B params

Tensor type

I32

BF16

Inference Providers NEW

Text Generation

This model is not currently available via any of the supported third-party Inference Providers, and the model is not deployed on the HF Inference API.

Model tree for minyichen/Llama-3-Taiwan-8B-Instruct-GPTQ

Base model

meta-llama/Meta-Llama-3-70B

Finetuned

yentinglin/Llama-3-Taiwan-8B-Instruct

Quantized

(15)

this model