|
--- |
|
license: cc-by-nc-sa-4.0 |
|
language: |
|
- ja |
|
base_model: |
|
- llm-jp/llm-jp-3-13b |
|
--- |
|
|
|
# Fine-tuned Japanese Instruction Model |
|
|
|
This is a fine-tuned version of the base model **[llm-jp/llm-jp-3-13b](https://huggingface.co/llm-jp/llm-jp-3-13b)** using the **ichikara-instruction** dataset. |
|
The model has been fine-tuned for **Japanese instruction-following tasks**. |
|
|
|
--- |
|
|
|
## Model Information |
|
|
|
### **Base Model** |
|
- **Model**: [llm-jp/llm-jp-3-13b](https://huggingface.co/llm-jp/llm-jp-3-13b) |
|
- **Architecture**: Causal Language Model |
|
- **Parameters**: 13 billion |
|
|
|
### **Fine-tuning Dataset** |
|
- **Dataset**: [ichikara-instruction](https://liat-aip.sakura.ne.jp/wp/llmのための日本語インストラクションデータ作成/) |
|
- **Authors**: 関根聡, 安藤まや, 後藤美知子, 鈴木久美, 河原大輔, 井之上直也, 乾健太郎 |
|
- **License**: [CC-BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/) |
|
|
|
The dataset includes Japanese instruction-response pairs and has been tailored for Japanese **instruction-following tasks**. |
|
|
|
関根聡, 安藤まや, 後藤美知子, 鈴木久美, 河原大輔, 井之上直也, 乾健太郎. ichikara-instruction: LLMのための日本語インストラクションデータの構築. 言語処理学会第30回年次大会(2024) |
|
|
|
--- |
|
|
|
## Usage |
|
|
|
1. Install Required Libraries |
|
|
|
``` |
|
!pip install -U bitsandbytes |
|
!pip install -U transformers |
|
!pip install -U accelerate |
|
!pip install -U datasets |
|
!pip install -U peft |
|
``` |
|
|
|
2. Load the Model and Libraries |
|
|
|
``` |
|
from transformers import ( |
|
AutoModelForCausalLM, |
|
AutoTokenizer, |
|
BitsAndBytesConfig, |
|
) |
|
from peft import PeftModel |
|
import torch |
|
|
|
# Hugging Face Token (recommended to set via environment variable) |
|
HF_TOKEN = "YOUR_HF_ACCESS_TOKEN" |
|
|
|
# Model and adapter IDs |
|
base_model_id = "llm-jp/llm-jp-3-13b" # Base model |
|
adapter_id = "sasakipeter/llm-jp-3-13b-finetune" |
|
|
|
# QLoRA (4-bit quantization) configuration |
|
bnb_config = BitsAndBytesConfig( |
|
load_in_4bit=True, |
|
bnb_4bit_quant_type="nf4", |
|
bnb_4bit_compute_dtype=torch.bfloat16, |
|
) |
|
``` |
|
|
|
3. Load the Base Model and LoRA Adapter |
|
|
|
``` |
|
# Load base model with 4-bit quantization |
|
model = AutoModelForCausalLM.from_pretrained( |
|
base_model_id, |
|
quantization_config=bnb_config, |
|
device_map="auto", |
|
token=HF_TOKEN |
|
) |
|
|
|
# Load tokenizer |
|
tokenizer = AutoTokenizer.from_pretrained( |
|
base_model_id, |
|
trust_remote_code=True, |
|
token=HF_TOKEN |
|
) |
|
|
|
# Integrate LoRA adapter into the base model |
|
model = PeftModel.from_pretrained(model, adapter_id, token=HF_TOKEN) |
|
``` |
|
|
|
4. Perform Inference |
|
|
|
``` |
|
# Example input prompt |
|
input_text = """次の文章を要約してください。 |
|
|
|
日本は四季があり、春には桜が咲き、夏には暑さが続きます。秋には紅葉が美しく、冬には雪が降ります。""" |
|
|
|
# Format the input prompt |
|
prompt = f"""### 指示 |
|
{input_text} |
|
### 回答 |
|
""" |
|
|
|
# Tokenize input and move to the model's device |
|
tokenized_input = tokenizer(prompt, return_tensors="pt").to(model.device) |
|
|
|
# Generate output |
|
with torch.no_grad(): |
|
outputs = model.generate( |
|
**tokenized_input, |
|
max_new_tokens=100, |
|
do_sample=False, |
|
repetition_penalty=1.2, |
|
pad_token_id=tokenizer.eos_token_id |
|
) |
|
|
|
# Decode the output |
|
output = tokenizer.decode(outputs[0][tokenized_input.input_ids.size(1):], skip_special_tokens=True) |
|
print("Output:") |
|
print(output) |
|
``` |
|
|
|
--- |
|
|
|
## License |
|
|
|
This model is released under the **CC-BY-NC-SA 4.0** license. |
|
|
|
- **Base Model**: [llm-jp/llm-jp-3-13b](https://huggingface.co/llm-jp/llm-jp-3-13b) (Apache License 2.0) |
|
- **Fine-Tuning Dataset**: ichikara-instruction (CC-BY-NC-SA 4.0) |
|
|
|
**Fine-tuned Model License**: |
|
Due to the Share-Alike (SA) condition of the ichikara-instruction dataset, the fine-tuned model is licensed under **CC-BY-NC-SA 4.0**. |
|
This means the model can only be used for **non-commercial purposes**, and any derivative works must adopt the same license. |
|
|
