lucyknada commited on
Commit
d02bf86
·
verified ·
1 Parent(s): 2282f56

Upload ./README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +67 -0
README.md ADDED
@@ -0,0 +1,67 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model:
3
+ - anthracite-org/magnum-v2-12b
4
+ - cgato/Nemo-12b-Humanize-SFT-v0.2-Quarter
5
+ library_name: transformers
6
+ tags:
7
+ - mergekit
8
+ - merge
9
+
10
+ ---
11
+ ### exl2 quant (measurement.json in main branch)
12
+ ---
13
+ ### check revisions for quants
14
+ ---
15
+
16
+ # output
17
+
18
+ This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
19
+
20
+ ## Merge Details
21
+ ### Merge Method
22
+
23
+ This model was merged using the [DARE TIES](https://arxiv.org/abs/2311.03099) merge method using [cgato/Nemo-12b-Humanize-SFT-v0.2-Quarter](https://huggingface.co/cgato/Nemo-12b-Humanize-SFT-v0.2-Quarter) as a base.
24
+
25
+ ### Models Merged
26
+
27
+ The following models were included in the merge:
28
+ * [anthracite-org/magnum-v2-12b](https://huggingface.co/anthracite-org/magnum-v2-12b)
29
+
30
+ ### Configuration
31
+
32
+ The following YAML configuration was used to produce this model:
33
+
34
+ ```yaml
35
+ base_model: cgato/Nemo-12b-Humanize-SFT-v0.2-Quarter
36
+ config_source: cgato/Nemo-12b-Humanize-SFT-v0.2-Quarter
37
+ dtype: float32
38
+ merge_method: dare_ties
39
+
40
+ parameters:
41
+ normalize: true
42
+ int8_mask: true
43
+ density: 0.9
44
+
45
+ tokenizer:
46
+ source: union
47
+ tokens:
48
+ <|im_start|>system:
49
+ source: "cgato/Nemo-12b-Humanize-SFT-v0.2-Quarter"
50
+ <|im_start|>assistant:
51
+ source: "cgato/Nemo-12b-Humanize-SFT-v0.2-Quarter"
52
+ <|im_start|>user:
53
+ source: "cgato/Nemo-12b-Humanize-SFT-v0.2-Quarter"
54
+ <|im_end|>:
55
+ source: "cgato/Nemo-12b-Humanize-SFT-v0.2-Quarter"
56
+
57
+ chat_template: "chatml"
58
+
59
+ models:
60
+ - model: anthracite-org/magnum-v2-12b
61
+ parameters:
62
+ weight: 0.4
63
+ - model: cgato/Nemo-12b-Humanize-SFT-v0.2-Quarter
64
+ parameters:
65
+ weight: 0.6
66
+
67
+ ```