|
--- |
|
license: apache-2.0 |
|
datasets: |
|
- OpenAssistant/oasst1 |
|
language: |
|
- en |
|
--- |
|
|
|
## 🐋 Humback |
|
|
|
The proposed Humback is a novel framework that can augment the instruction data for supervised fine-tuning with high quality. |
|
|
|
This is a backward model $M_{yx}$ for [Humback](https://arxiv.org/pdf/2308.06259.pdf) reproduction. |
|
|
|
This model is trained on the seed data in a reversed order (generate instruction given response). |
|
|
|
The seed data is a sampled dataset from [oasst1](https://huggingface.co/datasets/OpenAssistant/oasst1). |
|
|
|
You may find more details and usage examples in [Spico197/Humback](https://github.com/Spico197/Humback) . |
|
|
|
## 📜 Reference |
|
|
|
```bibtex |
|
@misc{li2023selfalignment, |
|
title={Self-Alignment with Instruction Backtranslation}, |
|
author={Xian Li and Ping Yu and Chunting Zhou and Timo Schick and Luke Zettlemoyer and Omer Levy and Jason Weston and Mike Lewis}, |
|
year={2023}, |
|
eprint={2308.06259}, |
|
archivePrefix={arXiv}, |
|
primaryClass={cs.CL} |
|
} |
|
``` |
