Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,54 @@
+---
+license: apache-2.0
+datasets:
+- stanfordnlp/SHP
+- Anthropic/hh-rlhf
+- OpenAssistant/oasst1
+language:
+- en
+metrics:
+- accuracy
+tags:
+- human feedback
+- rlhf
+- preferences
+- alignment
+- HALO
+- halos
+- dpo
+- rl
+---
+
+![halos](https://gist.github.com/assets/29318529/fe2d8391-dbd1-4b7e-9dc4-7cb97e55bc06)
+
+This repo contains the model checkpoints for:
+- model family <b>pythia2-8b</b>
+- optimized with the loss <b>CSFT</b>
+- aligned using the SHP, Anthropic HH and Open Assistant datasets.
+
+To prompt Archangel models, ensure that the format is consistent with that of TuluV2.
+For example, a prompt should be formatted as follows, where `<|user|>` corresponds to the human's role and `<|assistant|>` corresponds to the LLM's role.
+The human should speak first:
+```
+<|user|>
+Hi! I'm looking for a cake recipe.
+<|assistant|>
+What kind of cake?
+<|user|>
+Chocolate cake.
+<|assistant|>
+```
+Note that a beginning-of-sequence (BOS) token is automatically added by all Archangel models during tokenization and does not have to be added by you. No end-of-sequence (EOS) token is added to the prompt.
+
+Please refer to our [code repository](https://github.com/ContextualAI/HALOs) or [blog](https://contextual.ai/better-cheaper-faster-llm-alignment-with-kto/) which contains intructions for training your own HALOs and links to our model cards.
+
+If you find this repo or the technical paper useful in your research, please feel free to cite [our work](https://github.com/ContextualAI/HALOs/blob/main/assets/report.pdf):
+```
+@techreport{ethayarajh2023halos,
+  author = {Ethayarajh, Kawin and Xu, Winnie, and Jurafsky, Dan and Kiela, Douwe},
+  title = {Human-Centered Loss Functions (HALOs)},
+  institution = {Contextual AI},
+  note = {https://github.com/ContextualAI/HALOs/blob/main/assets/report.pdf},
+  year = {2023},
+}
+```