Hugging Face's logo

emmabhl
/
Qwen1.5-0.5B-Chat-ORCA-cDPO

Text Generation
PEFT
Safetensors
English
chat
conversational
Model card Files Community
Qwen1.5-0.5B-Chat-ORCA-cDPO / README.md
emmabhl's picture
emmabhl
Update README.md
a08f93c verified
preview code
|
raw
history blame contribute delete
1.05 kB
metadata
library_name: peft
base_model: Qwen/Qwen1.5-0.5B-Chat
language:
  - en
pipeline_tag: text-generation
tags:
  - chat
widget:
  - text: What is the sum of the first 10 positive integers?

Qwen1.5-0.5B-Chat with EPFL DPO fine-tuning

Qwen1.5-0.5B-Chat DPO fine-tuned on the Orca Math dataset that consists of ~200K grade school math word problems

Model Details

Model Description

The model was developed during the course Modern Natural Language Processing (CS-552). Its aim is to fine-tune the base model (Qwen/Qwen1.5-0.5B-Chat) to accurately answer open-ended and multiple-choice questions from Orca Math dataset

  • Developed by: Emma Lise Boehly, Ahmed Aziz Ben Haj Hmida and Jan Kokla
  • Finetuned from model: Qwen/Qwen1.5-0.5B-Chat

Training Details

Training Data

HuggingFace dataset : microsoft/orca-math-word-problems-200k

Training Procedure

Training Hyperparameters

  • Training regime: cDPO with bf16 mixed precision, $\beta=0.2$, $lr=3 \times 10^{-6}$, and $label_smoothing=0.2$

  • PEFT 0.10.0