Update README.md
Browse files
README.md
CHANGED
|
@@ -19,6 +19,8 @@ This model is [Openchat-3.5-0106](https://huggingface.co/openchat/openchat-3.5-0
|
|
| 19 |
|
| 20 |
The model was fine-tuned using [Rank-Stabilized LoRA](https://huggingface.co/blog/damjan-k/rslora) and the [LongAlpaca-12K](Yukang/LongAlpaca-12k) dataset. I hope to continue extending the context in future versions and then apply the same methods to my [upscaled versions of OpenChat-3.5](https://huggingface.co/collections/Pretergeek/openchat-35-0106-with-additional-layers-66a8d3262c7c3ebdd7783a29) that were created using Block Expansion instead of Depth UP Scaling.
|
| 21 |
|
|
|
|
|
|
|
| 22 |
## Citations
|
| 23 |
```
|
| 24 |
@misc{zhu2024poseefficientcontextwindow,
|
|
|
|
| 19 |
|
| 20 |
The model was fine-tuned using [Rank-Stabilized LoRA](https://huggingface.co/blog/damjan-k/rslora) and the [LongAlpaca-12K](Yukang/LongAlpaca-12k) dataset. I hope to continue extending the context in future versions and then apply the same methods to my [upscaled versions of OpenChat-3.5](https://huggingface.co/collections/Pretergeek/openchat-35-0106-with-additional-layers-66a8d3262c7c3ebdd7783a29) that were created using Block Expansion instead of Depth UP Scaling.
|
| 21 |
|
| 22 |
+
After fine-tuning, the model was tested using passkey retrieval and achieved a score of 100%.
|
| 23 |
+
|
| 24 |
## Citations
|
| 25 |
```
|
| 26 |
@misc{zhu2024poseefficientcontextwindow,
|