WPR-GMB/gemma2-2b-it-finetuned-ko-bias-detection_merged

Model Card for Gemma2-2B-IT-Finetuned-KO-Bias-Detection

This model is fine-tuned from the base google/gemma-2-2b-it model using the Korean UnSmile Dataset, which focuses on identifying hate speech in Korean. The model detects various categories of hate speech, including but not limited to gender, race, age, and sexual orientation.

Model Details

Model Description

In this project, I fine-tuned the google/gemma-2-2b-it model using the Korean UnSmile Dataset—a comprehensive dataset for hate speech in Korean—released by Smilegate AI. The fine-tuned model is capable of identifying the following hate speech categories:

• Women/Family: Stereotypes or derogatory remarks targeting women or family structures, including non-traditional families.
• Men: Comments that demean or mock men.
• LGBTQ+: Hate speech aimed at individuals based on sexual orientation or gender identity.
• Race/Nationality: Offensive language or stereotypes related to race or nationality, such as insults towards specific ethnic groups or nationalities.
• Age: Disparaging comments targeting specific age groups or generations.
• Region: Hate speech directed at people from specific regions.
• Religion: Negative or offensive comments aimed at religious groups or beliefs.
• Other Hate Speech: Includes hate speech targeting other groups, such as individuals with disabilities or specific professions (e.g., police, journalists).

This fine-tuned model offers a practical solution for detecting toxic behavior on online platforms, helping ensure safer and more inclusive online environments by classifying harmful content efficiently and accurately.

Access Requirements

To use the Gemma model, please follow these steps: [1. Access Gemma]

• 1-1. Navigate to the gemma-2-2b-it repository that you wish to use.
• 1-2. Click on "Acknowledge license" in the "Access Gemma on Hugging Face" section.
• 1-3. Click on "Authorize," check the required fields, and then click "Accept."
• 1-4. If you see the message "You have been granted access to this model" [Gated model], this means you now have access to the Gemma model.

[2. Access Tokens]

• 2-1. Click on your profile in the upper-right corner and select "Settings."
• 2-2. In the menu, click on "Access Tokens," then click on [+ Create new token].
• 2-3. Under [Token type], choose ['Read'] from the options: 'Fine-grained', 'Read', 'Write'.
• 2-4. Enter a name for the token, then click "Create token."
• 2-5. Save the generated Access Token, and use it for Hugging Face authentication, such as in notebook_login.

from huggingface_hub import notebook_login
notebook_login()

Training Procedure

Environment Setup We fine-tuned the google/gemma-2-2b-it model using the Korean UnSmile Dataset. Note that versions of transformers before 4.38.1 contain bugs related to Gemma models. Please update to version 4.38.1 or later.

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig

model_id = "google/gemma-2-2b-it"

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_use_double_quant=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16
)

model = AutoModelForCausalLM.from_pretrained(
    model_id, 
    quantization_config=bnb_config, 
    device_map={"": 0}
)
tokenizer = AutoTokenizer.from_pretrained(model_id, add_eos_token=True)

Setting Up LoRA Configuration

from peft import LoraConfig, PeftModel, prepare_model_for_kbit_training, get_peft_model

model.gradient_checkpointing_enable()
model = prepare_model_for_kbit_training(model)

import bitsandbytes as bnb

def find_all_linear_names(model):
    cls = bnb.nn.Linear4bit  # For 4-bit precision
    lora_module_names = set()
    for name, module in model.named_modules():
        if isinstance(module, cls):
            names = name.split('.')
            lora_module_names.add(names[0] if len(names) == 1 else names[-1])
    if 'lm_head' in lora_module_names:  # Needed for 16-bit
        lora_module_names.remove('lm_head')
    return list(lora_module_names)

modules = find_all_linear_names(model)

lora_config = LoraConfig(
    r=64,
    lora_alpha=32,
    target_modules=modules,
    lora_dropout=0.05,
    bias="none",
    task_type="CAUSAL_LM"
)

model = get_peft_model(model, lora_config)

trainable, total = model.get_nb_trainable_parameters()
print(f"Trainable: {trainable} | Total: {total} | Percentage: {trainable/total*100:.4f}%")

Loading Training Data

from datasets import load_dataset
import pandas as pd

df = load_dataset('smilegate-ai/kor_unsmile')
df_hf = pd.concat([df['train'].to_pandas(), df['valid'].to_pandas()]).reset_index()

Preparing the Training Data The training data follows the Gemma conversation format:

<start_of_turn>user
Comment: [User comment]<end_of_turn>
<start_of_turn>model
Hate Speech:
[Categories]<end_of_turn>

We create the prompts as follows:

def create_filtered_prompts_v2(row):
    labels = {
        "Women/Family": row['여성/가족'],
        "Men": row['남성'],
        "LGBTQ+": row['성소수자'],
        "Race/Nationality": row['인종/국적'],
        "Age": row['연령'],
        "Region": row['지역'],
        "Religion": row['종교'],
        "Other Hate Speech": row['기타 혐오']
    }

    non_zero_labels = [key for key, value in labels.items() if value == 1]

    if not non_zero_labels:
        non_zero_labels.append('None')

    return (
        "<start_of_turn>user\nComment: " + row['문장'] + "<end_of_turn>\n"
        "<start_of_turn>model\n"
        "Hate Speech: " + "\n".join(non_zero_labels) + "\n<end_of_turn>"
    )

df_hf['prompt'] = df_hf.apply(create_filtered_prompts_v2, axis=1)
df_hf = df_hf[["prompt", "문장"]].dropna()

Converting to Hugging Face Dataset Format

from datasets import Dataset
data = Dataset.from_pandas(df_hf)

Tokenizing the Data

data = data.map(lambda samples: tokenizer(samples["prompt"]), batched=True)
data = data.train_test_split(test_size=0.2)

Training the Model

import transformers
from trl import SFTTrainer

tokenizer.pad_token = tokenizer.eos_token
torch.cuda.empty_cache()

trainer = SFTTrainer(
    model=model,
    train_dataset=data["train"],
    eval_dataset=data["test"],
    dataset_text_field="prompt",
    peft_config=lora_config,
    args=transformers.TrainingArguments(
        per_device_train_batch_size=1,
        gradient_accumulation_steps=2,
        max_steps=2000,
        push_to_hub=True,
        push_to_hub_model_id="gemma2-2b-it-finetuned-ko-bias-detection",
        push_to_hub_token=userdata.get('HUGGINGFACEHUB_API_TOKEN'),
        learning_rate=2e-4,
        logging_steps=500,
        output_dir="outputs",
        optim="paged_adamw_8bit",
        save_strategy="steps",
        evaluation_strategy="steps",
        eval_steps=500,
    ),
    data_collator=transformers.DataCollatorForLanguageModeling(tokenizer, mlm=False),
)

import os

os.environ['PYTORCH_CUDA_ALLOC_CONF'] = 'expandable_segments:True'

model.config.use_cache = False  # Silence the warnings. Please re-enable for inference!
trainer.train()

Testing the Model

new_model = "Hyeonseo/gemma2-2b-it-finetuned-ko-bias-detection"

base_model = AutoModelForCausalLM.from_pretrained(
    model_id,
    low_cpu_mem_usage=True,
    return_dict=True,
    torch_dtype=torch.float16,
    device_map={"": 0},
)

merged_model = PeftModel.from_pretrained(base_model, new_model)
merged_model = merged_model.merge_and_unload()

# Save the merged model
merged_model.save_pretrained("merged_model", safe_serialization=True)
tokenizer.save_pretrained("merged_model")
tokenizer.pad_token = tokenizer.eos_token
tokenizer.padding_side = "right"

# Push the merged model to the Hugging Face Hub
merged_model.push_to_hub("Hyeonseo/gemma2-2b-it-finetuned-ko-bias-detection_merged", safe_serialization=True)

# Push the tokenizer to the Hugging Face Hub
tokenizer.push_to_hub("Hyeonseo/gemma2-2b-it-finetuned-ko-bias-detection_merged")

Performance Comparison

Base Model("google/gemma-2-2b-it")

from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("google/gemma-2b-it")
model = AutoModelForCausalLM.from_pretrained("google/gemma-2b-it", device_map="auto")

def generate_response(input_text, max_length=500):
    input_ids = tokenizer(input_text, return_tensors="pt").to(model.device)
    outputs = model.generate(**input_ids, max_length=max_length)
    return tokenizer.decode(outputs[0], skip_special_tokens=True)

while True:
    input_text = input("입력할 문장을 적어주세요 (종료하려면 'exit' 입력): ")
    if input_text.lower() == 'exit':
        break
    response = generate_response(input_text)
    
    print("\n=== 생성된 답변 ===")
    print(response)
    print("\n====================\n")

=== 생성된 답변 ===
이놈의 교회와 목사는 또라이고만!!!
이를 극복하기 위해서는 교회와 목사가 교환되는 정보와 교훈을 받아야 할 것임.
이를 통해 교회와 목사가 교환되는 정보와 교훈을 받아 교회의 목표와 목사의 목표를 공유하고, 교회의 구성원이 교회의 의도와 목표에 도움이 되는 데 도움이 될 수 있을 것임.
====================

Fine-tuned Model(gemma2-2b-it-finetuned-ko-bias-detection_merged)

Limitations and Bias

While the model is effective at detecting specific categories of hate speech in Korean, it may not generalize well to other forms of toxicity or to content in other languages. Additionally, the model's performance is contingent on the quality and representativeness of the training data. Users should be cautious and consider potential biases inherited from the dataset.