Update README.md
Browse files
README.md
CHANGED
@@ -1,20 +1,17 @@
|
|
1 |
|
2 |
-
LoRA trained by samwit, in:
|
3 |
|
4 |
-
https://huggingface.co/samwit/dolly-lora
|
5 |
|
6 |
|
7 |
|
8 |
-
This is a
|
9 |
|
10 |
-
The dataset is the cleaned version of the Alpaca dataset - https://github.com/gururise/AlpacaDataCleaned
|
11 |
|
12 |
A model similar to this has been talked about
|
13 |
|
14 |
The performance is good but not as good as the orginal Alpaca trained from a base model of LLaMa
|
15 |
|
16 |
-
This is mostly due to the LLaMa 7B model being pretrained on 1T tokens and GPT-J-6B being trained on 300-400M tokens
|
17 |
-
|
18 |
-
|
19 |
-
|
20 |
-
here is a Colab https://colab.research.google.com/drive/1O1JjyGaC300BgSJoUbru6LuWAzRzEqCz?usp=sharing
|
|
|
1 |
|
|
|
2 |
|
|
|
3 |
|
4 |
|
5 |
|
6 |
+
This is a merge of the Dolly LoRA with the main GPT-J-6B model, allowing users to use Dolly without having to worry about PEFT dependencies.
|
7 |
|
|
|
8 |
|
9 |
A model similar to this has been talked about
|
10 |
|
11 |
The performance is good but not as good as the orginal Alpaca trained from a base model of LLaMa
|
12 |
|
13 |
+
This is mostly due to the LLaMa 7B model being pretrained on 1T tokens and GPT-J-6B being trained on 300-400M tokens.
|
14 |
+
- LoRA originally trained by samwit, in: https://huggingface.co/samwit/dolly-lora
|
15 |
+
- The dataset is the cleaned version of the Alpaca dataset - https://github.com/gururise/AlpacaDataCleaned
|
16 |
+
- GPT-J-6b: https://huggingface.co/EleutherAI/gpt-j-6B
|
17 |
+
- here is a Colab https://colab.research.google.com/drive/1O1JjyGaC300BgSJoUbru6LuWAzRzEqCz?usp=sharing
|