Update README.md
Browse files
README.md
CHANGED
@@ -5,6 +5,8 @@ tags:
|
|
5 |
- mergekit
|
6 |
- lazymergekit
|
7 |
- tiiuae/falcon-11B
|
|
|
|
|
8 |
---
|
9 |
|
10 |
# Falcon-5.5B-Dutch
|
@@ -25,4 +27,10 @@ slices:
|
|
25 |
layer_range: [56,59]
|
26 |
|
27 |
merge_method: passthrough
|
28 |
-
dtype: bfloat16\```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
5 |
- mergekit
|
6 |
- lazymergekit
|
7 |
- tiiuae/falcon-11B
|
8 |
+
language:
|
9 |
+
- nl
|
10 |
---
|
11 |
|
12 |
# Falcon-5.5B-Dutch
|
|
|
27 |
layer_range: [56,59]
|
28 |
|
29 |
merge_method: passthrough
|
30 |
+
dtype: bfloat16\```
|
31 |
+
|
32 |
+
PruneMe has been optimized using the AgentWaller/dutch-oasst1 dataset by investigating layer similarity with 4000 samples. The layer ranges for pruning were determined based on this analysis to maintain performance while reducing model size.
|
33 |
+
|
34 |
+
![image/png](https://cdn-uploads.huggingface.co/production/uploads/660c0a02cf274b3ab77dd6b7/PF3SzEhQRJPXyYi2KqS1A.png)
|
35 |
+
|
36 |
+
Note: This is a base language model and has not been optimized for conversational or chat applications. Further fine-tuning may be required to adapt it for specific use cases.
|