r-yuba62/llm-jp-3-13b-finetune

Summary

This model was created as part of the creation of a submitted model for the competition of [the Matsuo Lab Large Scale Language Modeling Course 2024] (https://weblab.t.u-tokyo.ac.jp/lecture/course-list/large-language-model/) .

Uploaded model

Developed by: r-yuba License: apache-2.0 Finetuned from model : llm-jp/llm-jp-3-13b

How to Usage

To Install Packages

!pip install -U transformers
!pip install -U bitsandbytes
!pip install -U accelerate
!pip install -U datasets
!pip install -U peft
!pip install -U trl==0.12.0

Methods of Inference

from transformers import (
    AutoModelForCausalLM,
    AutoTokenizer,
    BitsAndBytesConfig,
)
from peft import PeftModel
import torch

HF_TOKEN = "AVAILABLE YOUR-HF-TOKEN"

model_name = "llm-jp/llm-jp-3-13b"
adapter_name = "r-yuba62/llm-jp-3-13b-finetune"

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
)

# モデルとトークナイザーをロード
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    quantization_config=bnb_config,
    device_map="auto",
    token=HF_TOKEN
)

tokenizer = AutoTokenizer.from_pretrained(
    model_name, 
    trust_remote_code=True, 
    token=HF_TOKEN
)

# PEFTアダプターを適用
model = PeftModel.from_pretrained(model, adapter_name, token=HF_TOKEN)

def get_response(input_text):
    prompt = f"""### 指示
    {input_text}
    ### 回答:
    """
    # トークナイズ処理
    tokenized_input = tokenizer(
        prompt,
        return_tensors="pt",
        padding=True,
        truncation=True,
        max_length=512
    )

    input_ids = tokenized_input["input_ids"].to(model.device)
    attention_mask = tokenized_input["attention_mask"].to(model.device)

    # モデル生成
    with torch.no_grad():
        outputs = model.generate(
            input_ids=input_ids,
            attention_mask=attention_mask,
            max_new_tokens=512,
            do_sample=False,
            repetition_penalty=1.2,
            pad_token_id=tokenizer.eos_token_id
        )

    # 出力のデコード
    output_text = tokenizer.decode(outputs[0], skip_special_tokens=True)
    return output_text

input_text = "xxxを教えてください"
response = get_response(input_text)
print(response)

Base Model

base_model:- llm-jp/llm-jp-3-13b

Instruction tuning

The models have been fine-tuned on the following datasets.

Language Dataset description
Japanese ichikara-instruction-003-001-1.json A manually constructed instruction dataset

データセット作成チーム: 関根聡, 安藤まや, 後藤美知子, 鈴木久美, 河原大輔, 井之上直也, 乾健太郎. ichikara-instruction: LLMのための日本語インストラクションデータの構築. 言語処理学会第30回年次大会(2024)

License

Apache License, Version 2.0

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model is not currently available via any of the supported Inference Providers.
The model cannot be deployed to the HF Inference API: The model has no pipeline_tag.