README.md

---
base_model: meta-llama/Llama-2-7b-hf
library_name: peft
---

# Model Card for Instruction Backtranslation (Backward Model)

## Model Overview

This repository contains a fine-tuned version of the Llama-2-7b-hf model specifically trained for **Instruction Backtranslation**, implementing the method described in the paper "Self Alignment with Instruction Backtranslation." The model is trained as a backward model to predict original instructions given their corresponding outputs. This reverse training aims to improve model alignment and self-consistency.

## Model Details

### Model Description

The model utilizes the LLaMA-2 architecture fine-tuned using Low-Rank Adaptation (LoRA) techniques (PEFT library). The primary goal is to reconstruct instructions (`x`) from outputs (`y`), thus creating pairs `(y, x)` for backward prediction.

- **Developed by:** Abhishek Sagar Sanda
- **Model type:** LoRA-finetuned Causal LM
- **Language(s):** English
- **License:** Apache-2.0 (consistent with base LLaMA-2 model)
- **Finetuned from:** [meta-llama/Llama-2-7b-hf](https://huggingface.co/meta-llama/Llama-2-7b-hf)

### Model Sources

- **Paper:** [Self Alignment with Instruction Backtranslation](https://arxiv.org/abs/2308.06259)
- **Base Model Repository:** [meta-llama/Llama-2-7b-hf](https://huggingface.co/meta-llama/Llama-2-7b-hf)

## Intended Uses

### Direct Use

- Generating original instructions from outputs for alignment purposes.
- Research in model alignment, self-consistency, and instruction-following behavior.

### Downstream Use

- Enhancing forward instruction-following models via self-alignment methods.
- Improving instruction tuning datasets by generating diverse instructions from desired outputs.

### Out-of-Scope Use

- This model is not suited for general question-answering or generic text generation tasks.
- Avoid using in contexts requiring high factual accuracy without additional verification.

## Training Data

The model was trained on the **OpenAssistant-Guanaco** training dataset, focusing on `(output, instruction)` pairs for backward prediction.

## Training Procedure

### Preprocessing

- Dataset pairs were inverted to use outputs (`y`) as input and instructions (`x`) as labels.
- Standard tokenization was applied using LLaMA's tokenizer.

### Training Hyperparameters

- **LoRA Rank (r):** 8
- **LoRA Alpha:** 32
- **LoRA Dropout:** 0.05
- **Target Modules:** `k_proj`, `q_proj`, `v_proj`, `o_proj`
- **Training Precision:** bf16 mixed precision

## Evaluation

Evaluation involved assessing the accuracy of generated instructions against the original instructions. Key metrics include BLEU, ROUGE, and qualitative human evaluations.

## Technical Specifications

### Model Architecture

- LLaMA-2 Transformer Architecture with PEFT LoRA Adaptation
- **Tokenizer:** LLaMA Tokenizer (`tokenizer_class`: LlamaTokenizer)
- **Maximum Sequence Length:** Practically unlimited (`model_max_length` is set very large)

### Hardware

- GPUs: NVIDIA A100 GPUs recommended
- Cloud Provider: AWS/GCP

## How to Use

Here's a quick start guide:

```python
from peft import PeftModel, PeftConfig
from transformers import AutoModelForCausalLM, AutoTokenizer

base_model_name = "meta-llama/Llama-2-7b-hf"
peft_model_name = "your_hf_model_path"

config = PeftConfig.from_pretrained(peft_model_name)
tokenizer = AutoTokenizer.from_pretrained(base_model_name)
model = AutoModelForCausalLM.from_pretrained(base_model_name)
model = PeftModel.from_pretrained(model, peft_model_name)

inputs = tokenizer("Output text goes here", return_tensors="pt")
outputs = model.generate(**inputs)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

## Environmental Impact

Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute).

- **Hardware Type:** NVIDIA GeForce RTX 4070 GPU
- **Hours Used:** 3hrs


## Citation

If you use this model, please cite:

```bibtex
@article{xu2023selfalignment,
  title={Self Alignment with Instruction Backtranslation},
  author={Xu, et al.},
  journal={arXiv preprint arXiv:2308.06259},
  year={2023}
}
```

## Model Card Author

- Abhishek Sagar Sanda

## Model Card Contact

- [email protected]

### Framework versions

- PEFT 0.15.1
- Transformers 4.38.1