File size: 3,401 Bytes
a13e318 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 |
---
license: apache-2.0
language:
- en
base_model:
- mistralai/Mistral-7B-Instruct-v0.3
pipeline_tag: text2text-generation
library_name: unsloth
---
# ModelCard for UnGPT-v1
## Model Details
- **Name:** UnGPT-v1
- **Foundation Model:** Mistral v0.3 (7B parameters)
- **Recommended Context Length:** 16k tokens
- **Fine-tuning Methodology:** LoRA-based training with Odds Ratio Preference Optimization method, using a combination of ebooks and synthetic data.
## Usage Instructions
Use the Alpaca format for prompts:
```
### Instruction:
{instruction}
### Input:
{input}
### Response:
```
**Example prompts**
For instructions, it is not recommended to deviate from the provided examples. For the input, a minimum is 10 sentences, but more can work as the model can handle longer context sizes (Thanks to the Mistral 7B v0.3 base model).
1. **Completion Prompt:**
```
### Instruction:
Continue writing the story while retaining writing style. Write about 10 sentences.
### Input:
It was a dark and stormy night...
### Response:
```
2. **Fill-in-the-middle Prompt:**
```
### Instruction:
Fill in the missing part of the story ({{FILL_ME}}) with about 10 sentences while retaining the writing style.
### Input:
The bus was speeding down the road, cops chasing after it.
{{FILL_ME}}
She woke up to find herself in an unfamiliar room...
### Response:
```
## Dataset Preparation
For dataset acquisition and cleanup please refer steps 1 and 2 of my text-completion example, [molbal/llm-text-completion-finetune](https://github.com/molbal/llm-text-completion-finetune/).
Chunking: Split texts into chunks based on sentence boundaries, aiming for 100 sentences per example.
- For completion examples, 90 sentences were used as input, 10 sentences as response.
- For fill-in-the-middle examples, 80 + 10 sentences as input (before and after the {{FILL_ME}} placeholder, respectively), and 10 sentences as response.
The beauty of the ORPO method is that for a single prompt we can set both a positive and a negative example. I wanted the model to avoid 'GPTisms' so I had gpt4o-mini generate answers both for completion and FOM tasks and added them as a neative example.
The dataset used is ~15k examples, each approximately 9000 characters long including input, accepted and refused response. (Note these are characters not tokens)
## Training setup
- Fine-tuned the Mistral v0.3 foundation model using Unsloth and ORPO trainer.
- Training configuration:
- Batch size: 1
- Gradient accumulation steps: 4
- Learning rate scheduler type: Linear
- Optimizer: AdamW (8-bit)
- Number of training epochs: 1
- Hardware
- I used GPU accelerated containers from the provider vast.ai (My referral link: https://cloud.vast.ai/?ref_id=123492 ) and executed training for ~8 hours on a single RTX 4090.
- Training costs
- ~5€ for renting a GPU pod (+15€ in unsuccessful attempts)
- ~5€ in OpenAI API costs for generating refusals
**Licensing and Citation**
- **License:** This model is licensed under the Apache License 2.0.
- **Citation:**
```bibtex
@misc{ungpt-v1,
author = Bálint Molnár-Kaló,
title = {UnGPT-v1: A Fine-tuned Mistral Model for Story Continuation},
howpublished = {\url{https://huggingface.co/models/molbal/UnGPT-v1}},
year = 2024
}
``` |