README.md

---
base_model: unsloth/tinyllama-chat-bnb-4bit
tags:
- text-generation-inference
- transformers
- unsloth
- llama
- trl
license: apache-2.0
language:
- en
---

# Fine-tuned TinyLlama for JSON Extraction

This repository contains a fine-tuned version of the `unsloth/tinyllama-chat-bnb-4bit` model, specifically trained for extracting product information from HTML snippets and outputting it in a JSON format.

## Model Details

- **Base Model:** `unsloth/tinyllama-chat-bnb-4bit`
- **Fine-tuning Method:** LoRA (Low-Rank Adaptation)
- **Trained on:** A custom dataset `json_extraction_dataset_500.json` of HTML product snippets and their corresponding JSON representations.

## Usage

This model can be used for tasks involving structured data extraction from HTML content.

### Loading the model

You can load the model and tokenizer using the `transformers` library:
```python
from unsloth import FastLanguageModel
import torch
import json

model_name = "learn-abc/html-model-tinyllama-chat-bnb-4bit" # Hugging face model repo ID
max_seq_length = 2048 # Or your chosen sequence length
dtype = None # Auto detection

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name = model_name,
    max_seq_length = max_seq_length,
    dtype = dtype,
    load_in_4bit = True,
)

FastLanguageModel.for_inference(model)

messages = [
    {"role": "user", "content": "Extract the product information:\n<div class='product'><h2>iPad Air</h2><span class='price'>$1344</span><span class='category'>audio</span><span class='brand'>Dell</span></div>"}
]

inputs = tokenizer.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_tensors="pt",
).to("cuda") # Or "cpu" if not using GPU

outputs = model.generate(
    input_ids=inputs,
    max_new_tokens=256,
    use_cache=True,
    temperature=0.7,
    do_sample=True,
    top_p=0.9,
)

response = tokenizer.batch_decode(outputs)[0]
print(response)
```

# Uploaded  model

- **Developed by:** learn-abc
- **License:** apache-2.0
- **Finetuned from model :** unsloth/tinyllama-chat-bnb-4bit

This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.

[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)