This is openchat/openchat-3.5-0106, tuned with DPO on a tiny subset Nectar. Only 200 steps, so nowhere close to a full epoch.

Careful attention was paid to make sure the chat template was followed properly.

Summary of versions:

openchat-nectar-0.1

  • 200 steps, no filtering on Nectar dataset, 5e-5 learning rate

openchat-nectar-0.2

  • empty repo, failed training. ignore it

openchat-nectar-0.3

  • 500 steps, no filtering on Nectar dataset, 5e-5 learning rate (same as 1 but with more steps)

openchat-nectar-0.4

  • 500 steps, filtered dataset to only include multi-chat-turn examples, used 4th ranking response as the "rejected" instead of 3rd, filtered out "good_natured=False", 5e-5 learning rate

openchat-nectar-0.5

  • 5000 steps (over a full epoch), filtered dataset to only include multi-chat-turn examples, used 4th ranking response as the "rejected" instead of 3rd, filtered out "good_natured=False", 5e-6 learning rate. Same as 0.4 but with 10x the steps, and 1/10th the learning rate

openchat-nectar-0.6

  • 500 steps, filtered dataset to only include multi-chat-turn examples, used 4th ranking response as the "rejected" instead of 3rd, filtered out "good_natured=False", 5e-5 learning rate. Same as 0.5 but with 1/10th the steps, and 10x the learning rate

Open LLM Leaderboard Evaluation Results

Detailed results can be found here

Metric Value
Avg. 69.94
AI2 Reasoning Challenge (25-Shot) 66.21
HellaSwag (10-Shot) 82.99
MMLU (5-Shot) 65.17
TruthfulQA (0-shot) 54.22
Winogrande (5-shot) 81.37
GSM8k (5-shot) 69.67
Downloads last month
31
Safetensors
Model size
7.24B params
Tensor type
FP16
·
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Model tree for andysalerno/openchat-nectar-0.1

Finetuned
(31)
this model
Quantizations
1 model

Dataset used to train andysalerno/openchat-nectar-0.1

Evaluation results