This repo contains the model and tokenizer checkpoints for:

[03/06/2024]: We are #2 on the (verified) Alpaca Eval 2.0 Leaderboard scoring 33.23!

To prompt this model, ensure that the format is consistent with that of TuluV2. For example, a prompt should be formatted as follows, where <|user|> corresponds to the human's role and <|assistant|> corresponds to the LLM's role. The human should speak first:


<|user|>
Hi! I'm looking for a cake recipe.
<|assistant|>
What kind of cake?
<|user|>
Chocolate cake.
<|assistant|>

Note that a beginning-of-sequence (BOS) token is automatically added at tokenization time and does not have to be added by you. No end-of-sequence (EOS) token is added to the prompt. You may also use our tokenizer's apply_chat_template if doing inference with chatml set or evaluating generations through non-local clients.

For more info on KTO refer to our code repository or blog for more details on the methodology.

If you found this work useful, feel free to cite our work:

@techreport{ethayarajh2023halos,
  author = {Ethayarajh, Kawin and Xu, Winnie, and Jurafsky, Dan and Kiela, Douwe},
  title = {Human-Centered Loss Functions (HALOs)},
  institution = {Contextual AI},
  note = {https://github.com/ContextualAI/HALOs/blob/main/assets/report.pdf},
  year = {2023},
}
Downloads last month
149
Safetensors
Model size
7.24B params
Tensor type
BF16
Β·
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Model tree for ContextualAI/Contextual_KTO_Mistral_PairRM

Merges
4 models
Quantizations
3 models

Dataset used to train ContextualAI/Contextual_KTO_Mistral_PairRM

Spaces using ContextualAI/Contextual_KTO_Mistral_PairRM 6