Update README.md
Browse files
README.md
CHANGED
@@ -18,7 +18,37 @@ tags:
|
|
18 |
---
|
19 |
# Joah-Llama-3-KoEn-8B-Coder-v2
|
20 |
|
21 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
22 |
### Merge Method
|
23 |
|
24 |
This model was merged using the [DARE](https://arxiv.org/abs/2311.03099) [TIES](https://arxiv.org/abs/2306.01708) merge method using [NousResearch/Meta-Llama-3-8B](https://huggingface.co/NousResearch/Meta-Llama-3-8B) as a base.
|
|
|
18 |
---
|
19 |
# Joah-Llama-3-KoEn-8B-Coder-v2
|
20 |
|
21 |
+
<a href="https://ibb.co/2Srsmn7"><img src="https://i.ibb.co/f9WnB1Y/Screenshot-2024-05-11-at-7-15-42-PM.png" alt="Screenshot-2024-05-11-at-7-15-42-PM" border="0"></a>
|
22 |
+
|
23 |
+
μ€λ λΆν° μλ‘μκ² λΉμ΄ λμ΄ μ€ μ¬λ¬λΆμ Merge Model
|
24 |
+
|
25 |
+
"μ’μ(Joah)" by AsianSoul
|
26 |
+
|
27 |
+
Soon Multi Language Model Merge based on this. First German Start (Korean / English / German) π
|
28 |
+
|
29 |
+
Where to use Joah : Medical, Korean, English, Translation, Code, Science... π₯
|
30 |
+
|
31 |
+
## π‘ Merge Details
|
32 |
+
|
33 |
+
|
34 |
+
The performance of this merge model doesn't seem to be bad though.-> Just opinion ^^ ποΈ
|
35 |
+
|
36 |
+
This may not be a model that satisfies you. But if we continue to overcome our shortcomings,
|
37 |
+
|
38 |
+
Won't we someday find the answer we want?
|
39 |
+
|
40 |
+
Don't worry even if you don't get the results you want.
|
41 |
+
|
42 |
+
I'll find the answer for you.
|
43 |
+
|
44 |
+
Soon real PoSE to extend Llama's context length to 64k with using my merge method : [reborn](https://medium.com/@puffanddmx82/reborn-elevating-model-adaptation-with-merging-for-superior-nlp-performance-f604e8e307b2)
|
45 |
+
|
46 |
+
I have found that most of merge's model outside so far do not actually have 64k in their configs. I will improve it in the next merge with my reborn. If that doesn't work, I guess I'll have to find another way, right?
|
47 |
+
|
48 |
+
256k is not possible. My computer is running out of memory.
|
49 |
+
|
50 |
+
If you support me, i will try it on a computer with maximum specifications, also, i would like to conduct great tests by building a network with high-capacity traffic and high-speed 10G speeds for you.
|
51 |
+
|
52 |
### Merge Method
|
53 |
|
54 |
This model was merged using the [DARE](https://arxiv.org/abs/2311.03099) [TIES](https://arxiv.org/abs/2306.01708) merge method using [NousResearch/Meta-Llama-3-8B](https://huggingface.co/NousResearch/Meta-Llama-3-8B) as a base.
|