This model is for the reproduction of results on Safe-RLHF dataset of paper "The crucial role of samplers in online direct preference optimization". You can download it and save it as "models/rlhflow_iter1", and then start the training pipeline. Since we've retrained the models, the results may slightly differ from that reported in the paper, and we will update it later.

Downloads last month
6
Safetensors
Model size
6.74B params
Tensor type
BF16
·
Inference API
Unable to determine this model's library. Check the docs .

Model tree for zhezi12138/alpaca-7b-iter-1

Finetuned
(2)
this model

Dataset used to train zhezi12138/alpaca-7b-iter-1