Chryslerx10/t5-small-amazon-reviews-summarization-finetuned-8bit-lora

Model Description

t5-small-amazon-reviews-summarization-finetuned-8bit-lora is a fine-tuned version of the T5-small model, adapted for the task of summarizing Amazon product reviews. The model has been fine-tuned using Low-Rank Adaptation (LoRA) techniques and 8-bit quantization, allowing for efficient training and inference while maintaining performance.

Model Architecture

• Base Model: T5-small
• Architecture: Encoder-Decoder Transformer
• Parameters: Approximately 60 million parameters
• Fine-tuning Method: LoRA (Low-Rank Adaptation) via the PEFT library
• Quantization: 8-bit quantization using the bitsandbytes library

Training Data

The model was fine-tuned on a dataset of Amazon product reviews on products of the beauty category, with corresponding summaries.

• Dataset Source: Custom dataset derived from Amazon product reviews
• Dataset Size: 19665 pairs
• Dataset Content: Pairs of product reviews and their corresponding summaries
• Dataset Split: 80% training, 20% validation

Training Procedure

Preprocessing

Input Preparation:
• Each input review was prefixed with "summarize: " to adhere to T5's format for summarization tasks.
• Inputs were tokenized with a maximum length of 2048 tokens.
Label Preparation:
• Summaries were tokenized with a maximum length of 512 tokens.
Tokenization:
• Used AutoTokenizer for T5-small.
• Applied truncation and padding to the maximum lengths.

Fine-tuning Configuration

LoRA Configuration:
    Rank (r): 8
    Alpha: 16
    Dropout: 0.5
    Task Type: SEQ_2_SEQ_LM (Sequence-to-Sequence Language Modeling)
Quantization Configuration:
    Load in 8-bit: True (using BitsAndBytesConfig)
Training Arguments:
    Output Directory: ./model/t5-small-amazon-review-summarization
    Evaluation Strategy: Every 1,000 steps
    Learning Rate: 2e-7
    Per Device Training Batch Size: 2
    Per Device Evaluation Batch Size: 8
    Max Steps: 40,000
    Logging Steps: 1,000
    Save Steps: 1,000
    Load Best Model at End: True
    Predict with Generate: True
    Generation Max Length: 512
    Generation Num Beams: 5

Training Environment

Hardware Used: Single NVIDIA RTX 4070 8GB
Frameworks and Libraries:
    Transformers
    Datasets
    PEFT
    BitsAndBytes
    Evaluate
    PyTorch

Metrics

The model was tested using BERTScorer to test the accuracy of the generated summaries for the reviews. Test set size of 3.8k pairs of reviews and summary.

• Precision: 0.87172
• Recall: 0.86239
• F1 score: 0.86686

Intended Use

The model is designed to generate concise summaries of Amazon product reviews. It can be utilized in applications such as:

• E-commerce Platforms: Summarizing customer reviews for quick insights.
• Product Analysis: Assisting businesses in understanding customer feedback.
• User Interfaces: Enhancing user experience by providing brief overviews of lengthy reviews.

How to Use

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM, BitsAndBytesConfig
from peft import LoraConfig, PeftModel

def generate_summary(text):
    inputs = tokenizer(
        text,
        return_tensors='pt',
        padding=True,
        truncation=True,
        max_length=2048,
    ).to(device)

    summary_ids = model.generate(
        input_ids=inputs['input_ids'],
        attention_mask=inputs['attention_mask'],
        max_length=512,
        top_k=5,
        top_p=0.95,
        temperature=0.7,
        num_return_sequences=10,
        no_repeat_ngram_size=2,
        do_sample=True,
    )

    summary = [tokenizer.decode(
        summary_id,
        skip_special_tokens=True,
        clean_up_tokenization_spaces=True,
    ) for summary_id in summary_ids]
    
    return summary

tokenizer = AutoTokenizer.from_pretrained("Chryslerx10/t5-small-amazon-reviews-summarization-finetuned-8bit-lora")
model = AutoModelForSeq2SeqLM.from_pretrained(
    "t5-small",
    device_map="auto",
)

model = PeftModel.from_pretrained(model, "Chryslerx10/t5-small-amazon-reviews-summarization-finetuned-8bit-lora", device_map='auto')

generate_summary("summarize: " + ".... reviews .....")

Limitations and Biases

• Data Bias: The model was trained on a specific dataset of Amazon reviews, which may not represent all product categories or customer perspectives.
• Language Support: The model primarily supports English and may not perform well on reviews in other languages.
• Generation Quality: While the model aims to produce coherent summaries, it may occasionally generate irrelevant or nonsensical text.