tykiww commited on
Commit
df0bf53
·
verified ·
1 Parent(s): cfbd01a

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +36 -2
README.md CHANGED
@@ -1,5 +1,5 @@
1
  ---
2
- title: Lora Instruction Training
3
  emoji: 🐨
4
  colorFrom: yellow
5
  colorTo: indigo
@@ -10,4 +10,38 @@ pinned: false
10
  license: apache-2.0
11
  ---
12
 
13
- Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ title: SLM Instruction Tuning using Unsloth
3
  emoji: 🐨
4
  colorFrom: yellow
5
  colorTo: indigo
 
10
  license: apache-2.0
11
  ---
12
 
13
+ # SLM Instruction Tuning using Unsloth
14
+
15
+ ### What?
16
+
17
+ This Gradio app is a simple interface to access [unsloth AI's](https://github.com/unslothai) fine-tuning methods but leveraging the A100 GPUs provided by [Huggingface Spaces](https://huggingface.co/docs/hub/en/spaces-overview). This outputs of this fine-tuning will be instruction tuned LoRA weights that will be uploaded into your personal huggingface models repository.
18
+
19
+ [<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
20
+
21
+ ### Why?
22
+
23
+ The goal of this demo is to show how you can tune your own language models leveraging industry standard compute and fine tuning methods using a simple point-and-click UI.
24
+
25
+ In addition, compute, even on Google Colab's free tier is tight even with a T4 and rate limits are uncertain. This makes the use of the A100s on this demo useful for a small added boost to compute performance. For those looking to reduce the costs associated with training datasets can pull down the spaces repository to train their models at speed for $9 on The Huggingface Pro License.
26
+
27
+ This is a demo and not a production application. This application is subject a demand queue.
28
+
29
+ ### How?
30
+
31
+ Just start by following the guide below:
32
+
33
+ 1) Flip to the Train Model tab at the top.
34
+ 2) Populate your username, repository, token, and model details
35
+ 3) Upload data from transformers or your local jsonl file. Please view [this guide](https://platform.openai.com/docs/guides/fine-tuning/preparing-your-dataset) for best practices.
36
+ 4) Eat a snack and wait as you train the model.
37
+
38
+ ### Coming soon!
39
+
40
+ - More models and added flexibility with guardrails on hyperparameter tuning.
41
+ - Downloads for a [WandB](https://wandb.ai/home) logger for training monitoring.
42
+
43
+ ### Other resources.
44
+
45
+ - [Unsloth's notebooks](https://colab.research.google.com/drive/1hhdhBa1j_hsymiW9m-WzxQtgqTH_NHqi?usp=sharing) to look at what is going on under the hood.
46
+
47
+