Update README.md
Browse files
README.md
CHANGED
@@ -1,5 +1,5 @@
|
|
1 |
---
|
2 |
-
title:
|
3 |
emoji: 🐨
|
4 |
colorFrom: yellow
|
5 |
colorTo: indigo
|
@@ -10,4 +10,38 @@ pinned: false
|
|
10 |
license: apache-2.0
|
11 |
---
|
12 |
|
13 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
---
|
2 |
+
title: SLM Instruction Tuning using Unsloth
|
3 |
emoji: 🐨
|
4 |
colorFrom: yellow
|
5 |
colorTo: indigo
|
|
|
10 |
license: apache-2.0
|
11 |
---
|
12 |
|
13 |
+
# SLM Instruction Tuning using Unsloth
|
14 |
+
|
15 |
+
### What?
|
16 |
+
|
17 |
+
This Gradio app is a simple interface to access [unsloth AI's](https://github.com/unslothai) fine-tuning methods but leveraging the A100 GPUs provided by [Huggingface Spaces](https://huggingface.co/docs/hub/en/spaces-overview). This outputs of this fine-tuning will be instruction tuned LoRA weights that will be uploaded into your personal huggingface models repository.
|
18 |
+
|
19 |
+
[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
|
20 |
+
|
21 |
+
### Why?
|
22 |
+
|
23 |
+
The goal of this demo is to show how you can tune your own language models leveraging industry standard compute and fine tuning methods using a simple point-and-click UI.
|
24 |
+
|
25 |
+
In addition, compute, even on Google Colab's free tier is tight even with a T4 and rate limits are uncertain. This makes the use of the A100s on this demo useful for a small added boost to compute performance. For those looking to reduce the costs associated with training datasets can pull down the spaces repository to train their models at speed for $9 on The Huggingface Pro License.
|
26 |
+
|
27 |
+
This is a demo and not a production application. This application is subject a demand queue.
|
28 |
+
|
29 |
+
### How?
|
30 |
+
|
31 |
+
Just start by following the guide below:
|
32 |
+
|
33 |
+
1) Flip to the Train Model tab at the top.
|
34 |
+
2) Populate your username, repository, token, and model details
|
35 |
+
3) Upload data from transformers or your local jsonl file. Please view [this guide](https://platform.openai.com/docs/guides/fine-tuning/preparing-your-dataset) for best practices.
|
36 |
+
4) Eat a snack and wait as you train the model.
|
37 |
+
|
38 |
+
### Coming soon!
|
39 |
+
|
40 |
+
- More models and added flexibility with guardrails on hyperparameter tuning.
|
41 |
+
- Downloads for a [WandB](https://wandb.ai/home) logger for training monitoring.
|
42 |
+
|
43 |
+
### Other resources.
|
44 |
+
|
45 |
+
- [Unsloth's notebooks](https://colab.research.google.com/drive/1hhdhBa1j_hsymiW9m-WzxQtgqTH_NHqi?usp=sharing) to look at what is going on under the hood.
|
46 |
+
|
47 |
+
|