metadata
license: llama2
base_model: ngoan/Llama-2-7b-vietnamese-20k
datasets:
- nthngdy/oscar-mini
- Tamnemtf/VietNamese_lang
language:
- vi
pipeline_tag: text-generation
tags:
- text-generation
- llama-2
- llama-2-7B
- llama2-vietnamese
- vietnamese
Model Details
- Model Name: llama-2-7b-vi-oscar_mini
- Purpose: Mục đích để train con model này để phục vụ việc học và đề tài nckh.
- Availability: The model checkpoint can be accessed on Hugging Face: Tamnemtf/llama-2-7b-vi-oscar_mini
- Model trên được train dựa trên model gốc là ngoan/Llama-2-7b-vietnamese-20k
How to Use
# Activate 4-bit precision base model loading
use_4bit = True
# Compute dtype for 4-bit base models
bnb_4bit_compute_dtype = "float16"
# Quantization type (fp4 or nf4)
bnb_4bit_quant_type = "nf4"
# Activate nested quantization for 4-bit base models (double quantization)
use_nested_quant = False
# Load the entire model on the GPU 0
device_map = {"": 0}
compute_dtype = getattr(torch, bnb_4bit_compute_dtype)
bnb_config = BitsAndBytesConfig(
load_in_4bit=use_4bit,
bnb_4bit_quant_type=bnb_4bit_quant_type,
bnb_4bit_compute_dtype=compute_dtype,
bnb_4bit_use_double_quant=use_nested_quant,
)
model = AutoModelForCausalLM.from_pretrained(
'Tamnemtf/llama-2-7b-vi-oscar_mini',
quantization_config=bnb_config,
device_map=device_map
)
model.config.use_cache = False
model.config.pretraining_tp = 1
tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
tokenizer.pad_token = tokenizer.eos_token
tokenizer.padding_side = "right" # Fix weird overflow issue with fp16 training
# Run text generation pipeline with our next model
prompt = "Canh chua cá lau là món gì ?"
pipe = pipeline(task="text-generation", model=model, tokenizer=tokenizer, max_length=200)
result = pipe(f"<s>[INST] {prompt} [/INST]")
print(result[0]['generated_text'])
Để ưu tiên cho việc dễ dàng tiếp cận với các sinh viên dưới đây là mẫu ví dụ chạy thử model trên colab bằng T4 https://colab.research.google.com/drive/1ME_k-gUKSY2NbB7GQRk3sqz56CKsSV5C?usp=sharing