Ttimofeyka commited on
Commit
750f02d
·
verified ·
1 Parent(s): ba12c6f

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -29
README.md CHANGED
@@ -7,33 +7,7 @@ tags:
7
  - merge
8
 
9
  ---
10
- # out
11
 
12
- This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
13
-
14
- ## Merge Details
15
- ### Merge Method
16
-
17
- This model was merged using the passthrough merge method.
18
-
19
- ### Models Merged
20
-
21
- The following models were included in the merge:
22
- * [TheDrummer/Cydonia-22B-v1.1](https://huggingface.co/TheDrummer/Cydonia-22B-v1.1)
23
-
24
- ### Configuration
25
-
26
- The following YAML configuration was used to produce this model:
27
-
28
- ```yaml
29
- slices:
30
- - sources:
31
- - model: TheDrummer/Cydonia-22B-v1.1
32
- layer_range: [0, 29]
33
- - sources:
34
- - model: TheDrummer/Cydonia-22B-v1.1
35
- layer_range: [41, 56]
36
- merge_method: passthrough
37
- dtype: bfloat16
38
-
39
- ```
 
7
  - merge
8
 
9
  ---
10
+ # Cydonia-18B
11
 
12
+ I reduced Cydonia v1.1 using merge cut so that it fits comfortably in video cards with 12 GB of video memory on the Q4_K_M quant.
13
+ I think an additional finetune is needed.