README.md

---
license: apache-2.0
datasets:
- OpenAssistant/oasst1
language:
- en
---

## 🐋 Humback

The proposed Humback is a novel framework that can augment the instruction data for supervised fine-tuning with high quality.

This is a backward model $M_{yx}$ for [Humback](https://arxiv.org/pdf/2308.06259.pdf) reproduction.

This model is trained on the seed data in a reversed order (generate instruction given response).

The seed data is a sampled dataset from [oasst1](https://huggingface.co/datasets/OpenAssistant/oasst1).

You may find more details and usage examples in [Spico197/Humback](https://github.com/Spico197/Humback) .

## 📜 Reference

```bibtex
@misc{li2023selfalignment,
    title={Self-Alignment with Instruction Backtranslation},
    author={Xian Li and Ping Yu and Chunting Zhou and Timo Schick and Luke Zettlemoyer and Omer Levy and Jason Weston and Mike Lewis},
    year={2023},
    eprint={2308.06259},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}
```