metadata

base_model: unsloth/tinyllama-chat-bnb-4bit
tags:
  - text-generation-inference
  - transformers
  - unsloth
  - llama
  - trl
license: apache-2.0
language:
  - en

Fine-tuned TinyLlama for JSON Extraction

This repository contains a fine-tuned version of the unsloth/tinyllama-chat-bnb-4bit model, specifically trained for extracting product information from HTML snippets and outputting it in a JSON format.

Model Details

Base Model: unsloth/tinyllama-chat-bnb-4bit
Fine-tuning Method: LoRA (Low-Rank Adaptation)
Trained on: A custom dataset json_extraction_dataset_500.json of HTML product snippets and their corresponding JSON representations.

Usage

This model can be used for tasks involving structured data extraction from HTML content.

Loading the model

You can load the model and tokenizer using the transformers library:

from unsloth import FastLanguageModel
import torch
import json

model_name = "learn-abc/html-model-tinyllama-chat-bnb-4bit" # Hugging face model repo ID
max_seq_length = 2048 # Or your chosen sequence length
dtype = None # Auto detection

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name = model_name,
    max_seq_length = max_seq_length,
    dtype = dtype,
    load_in_4bit = True,
)

FastLanguageModel.for_inference(model)

messages = [
    {"role": "user", "content": "Extract the product information:\n<div class='product'><h2>iPad Air</h2><span class='price'>$1344</span><span class='category'>audio</span><span class='brand'>Dell</span></div>"}
]

inputs = tokenizer.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_tensors="pt",
).to("cuda") # Or "cpu" if not using GPU

outputs = model.generate(
    input_ids=inputs,
    max_new_tokens=256,
    use_cache=True,
    temperature=0.7,
    do_sample=True,
    top_p=0.9,
)

response = tokenizer.batch_decode(outputs)[0]
print(response)

Uploaded model

Developed by: learn-abc
License: apache-2.0
Finetuned from model : unsloth/tinyllama-chat-bnb-4bit

This llama model was trained 2x faster with Unsloth and Huggingface's TRL library.