---
base_model: unsloth/meta-llama-3.1-8b-bnb-4bit
library_name: peft
---

# Model Card for Model ID

<!-- Provide a quick summary of what the model is/does. -->



## Model Details

### Model Description

<!-- Provide a longer summary of what this model is. -->



- **Developed by:** thepinkdrummer


## Direct Use

### Login
from huggingface_hub import notebook_login
notebook_login()

### Import the Models
from peft import PeftModel, PeftConfig
from transformers import AutoModelForCausalLM, AutoTokenizer

##### Load the base model and tokenizer
base_model_name = "unsloth/meta-llama-3.1-8b-bnb-4bit"
base_model = AutoModelForCausalLM.from_pretrained(base_model_name)
tokenizer = AutoTokenizer.from_pretrained(base_model_name)

##### Load PEFT configuration and the fine-tuned model
peft_model_name = "thepinkdrummer/maayavi"
config = PeftConfig.from_pretrained(peft_model_name)
model = PeftModel.from_pretrained(base_model, peft_model_name)

### Run Chat Model


import torch

def chat_with_model(prompt, max_length=512):
    inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
    
    with torch.no_grad():
        output = model.generate(
            inputs['input_ids'],
            max_length=max_length,
            num_return_sequences=1,
            temperature=0.7,
            top_p=0.9,
            top_k=50,
            do_sample=True,
            pad_token_id=tokenizer.eos_token_id,
            eos_token_id=tokenizer.eos_token_id
        )
    response = tokenizer.decode(output[0], skip_special_tokens=True)
    return response
    
user_input = "Hello! How are you?"
response = chat_with_model(user_input)
print(response)


### Framework versions
- PEFT 0.13.2