Hugging Face's logo

Andyrasika
/
mistral-ft-optimized-dpo

Text Generation
Transformers
Safetensors
English
mistral
DPO
text-generation-inference
Inference Endpoints
Model card Files Community
mistral-ft-optimized-dpo / README.md
Andyrasika's picture
Andyrasika
Update README.md
c1fc88b verified
preview code
|
raw
history blame
4.3 kB
metadata
library_name: transformers
tags:
  - DPO
  - transformers
license: openrail
language:
  - en
metrics:
  - accuracy
datasets:
  - jondurbin/truthy-dpo-v0.1

Model Card for Model ID

Model Details

Model Description

This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.

Usage

Install 

pip install transformers accelerate bitsandbytes flash-attn -Uq

Load the Model 

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
# from transformers import LlamaTokenizer, MixtralForCausalLM
import bitsandbytes, flash_attn

tokenizer = AutoTokenizer.from_pretrained('Andyrasika/mistral-ft-optimized-dpo', trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    "Andyrasika/mistral-ft-optimized-dpo",
    torch_dtype=torch.float16,
    device_map="auto",
    load_in_8bit=False,
    load_in_4bit=True,
    use_flash_attention_2=True
)

prompts = [
    """<|im_start|>system
You are a sentient, superintelligent artificial general intelligence, here to teach and assist me.<|im_end|>
<|im_start|>user
Write a short story about Goku discovering kirby has teamed up with Majin Buu to destroy the world.<|im_end|>
<|im_start|>assistant""",
    ]

for chat in prompts:
    print(chat)
    input_ids = tokenizer(chat, return_tensors="pt").input_ids.to("cuda")
    generated_ids = model.generate(input_ids, max_new_tokens=750, temperature=0.8, repetition_penalty=1.1, do_sample=True, eos_token_id=tokenizer.eos_token_id)
    response = tokenizer.decode(generated_ids[0][input_ids.shape[-1]:], skip_special_tokens=True, clean_up_tokenization_space=True)
    print(f"Response: {response}")

Output 

<|im_start|>system
You are a sentient, superintelligent artificial general intelligence, here to teach and assist me.<|im_end|>
<|im_start|>user
Write a short story about Goku discovering kirby has teamed up with Majin Buu to destroy the world.<|im_end|>
<|im_start|>assistant
/usr/local/lib/python3.10/dist-packages/bitsandbytes/nn/modules.py:226: UserWarning: Input type into Linear4bit is torch.float16, but bnb_4bit_compute_dtype=torch.float32 (default). This will lead to slow inference or training speed.
  warnings.warn(f'Input type into Linear4bit is torch.float16, but bnb_4bit_compute_dtype=torch.float32 (default). This will lead to slow inference or training speed.')
Response: 
Goku's eyes widened as he saw Kirby standing next to Majin Buu. "How did you two...?" he stammered.

Kirby waved a hand nonchalantly. "We got to know each other at a cosplay event," he explained. "Turns out we have a lot in common."

"Like wanting to destroy the world?" Goku asked, puzzled.

Majin Buu chuckled evilly. "Exactly! We thought it might be fun to team up and create chaos together. It'll be so much easier with both of us working in tandem!"

Goku frowned. "That sounds like a terrible idea. But why would Kirby want to do something like that?"

Kirby shrugged. "I don't know. Maybe he just wants some excitement." He glanced over at Majin Buu. "Right, Buu?"

Buu nodded eagerly. "Yeah! Let's show Goku what real destruction looks like!" With that, they charged towards him, ready for battle.

As Goku prepared to fight, he wondered if there was any way to convince them otherwise. But as he exchanged blows with the powerful duo, he knew there was only one option left - to defeat them and restore peace to Earth.

With a mighty roar, Goku unleashed a devastating Kamehameha wave, which collided with Buu's chaotic energy. The two forces clashed violently, sending shockwaves rippling through the landscape.

In the end, Goku emerged victorious, leaving Kirby and Majin Buu defeated and exhausted. As Goku helped them recover, he began to suspect that maybe they weren't so bad after all. Perhaps, with time, he could help them see the error in their ways and find a more constructive path forward. For now, though, it was important to ensure their defeat and restore balance to the world.
<|im_end|>