bunnycore commited on
Commit
570d8eb
·
verified ·
1 Parent(s): 892898f

Upload folder using huggingface_hub

Browse files
README.md CHANGED
@@ -1,7 +1,10 @@
1
  ---
2
  base_model:
3
  - bunnycore/Qwen2.5-7B-MixStock-V0.1
4
- - bunnycore/Qwen-2.5-7b-rp-lora
 
 
 
5
  library_name: transformers
6
  tags:
7
  - mergekit
@@ -15,25 +18,35 @@ This is a merge of pre-trained language models created using [mergekit](https://
15
  ## Merge Details
16
  ### Merge Method
17
 
18
- This model was merged using the Passthrough merge method using [bunnycore/Qwen2.5-7B-MixStock-V0.1](https://huggingface.co/bunnycore/Qwen2.5-7B-MixStock-V0.1) + [bunnycore/Qwen-2.5-7b-rp-lora](https://huggingface.co/bunnycore/Qwen-2.5-7b-rp-lora) as a base.
19
 
20
  ### Models Merged
21
 
22
  The following models were included in the merge:
23
-
 
 
 
24
 
25
  ### Configuration
26
 
27
  The following YAML configuration was used to produce this model:
28
 
29
  ```yaml
30
-
31
-
32
- base_model: bunnycore/Qwen2.5-7B-MixStock-V0.1+bunnycore/Qwen-2.5-7b-rp-lora
33
- dtype: bfloat16
34
- merge_method: passthrough
35
  models:
36
- - model: bunnycore/Qwen2.5-7B-MixStock-V0.1+bunnycore/Qwen-2.5-7b-rp-lora
37
- tokenizer_source: bunnycore/Qwen2.5-7B-MixStock-V0.1
 
 
 
 
 
 
 
 
 
 
 
 
38
 
39
  ```
 
1
  ---
2
  base_model:
3
  - bunnycore/Qwen2.5-7B-MixStock-V0.1
4
+ - bunnycore/Qwen2.5-7B-RRP-1M
5
+ - nvidia/AceInstruct-7B
6
+ - open-r1/OpenR1-Qwen-7B
7
+ - open-thoughts/OpenThinker-7B
8
  library_name: transformers
9
  tags:
10
  - mergekit
 
18
  ## Merge Details
19
  ### Merge Method
20
 
21
+ This model was merged using the [SCE](https://arxiv.org/abs/2408.07990) merge method using [bunnycore/Qwen2.5-7B-RRP-1M](https://huggingface.co/bunnycore/Qwen2.5-7B-RRP-1M) as a base.
22
 
23
  ### Models Merged
24
 
25
  The following models were included in the merge:
26
+ * [bunnycore/Qwen2.5-7B-MixStock-V0.1](https://huggingface.co/bunnycore/Qwen2.5-7B-MixStock-V0.1)
27
+ * [nvidia/AceInstruct-7B](https://huggingface.co/nvidia/AceInstruct-7B)
28
+ * [open-r1/OpenR1-Qwen-7B](https://huggingface.co/open-r1/OpenR1-Qwen-7B)
29
+ * [open-thoughts/OpenThinker-7B](https://huggingface.co/open-thoughts/OpenThinker-7B)
30
 
31
  ### Configuration
32
 
33
  The following YAML configuration was used to produce this model:
34
 
35
  ```yaml
 
 
 
 
 
36
  models:
37
+ # Pivot model
38
+ - model: bunnycore/Qwen2.5-7B-RRP-1M
39
+ # Target models
40
+ - model: open-thoughts/OpenThinker-7B
41
+ - model: open-r1/OpenR1-Qwen-7B
42
+ - model: bunnycore/Qwen2.5-7B-MixStock-V0.1
43
+ - model: nvidia/AceInstruct-7B
44
+ merge_method: sce
45
+ base_model: bunnycore/Qwen2.5-7B-RRP-1M
46
+ parameters:
47
+ select_topk: 0.65
48
+ int8_mask: true
49
+ dtype: bfloat16
50
+
51
 
52
  ```
config.json CHANGED
@@ -1,5 +1,5 @@
1
  {
2
- "_name_or_path": "bunnycore/Qwen2.5-7B-MixStock-V0.1",
3
  "architectures": [
4
  "Qwen2ForCausalLM"
5
  ],
 
1
  {
2
+ "_name_or_path": "bunnycore/Qwen2.5-7B-RRP-1M",
3
  "architectures": [
4
  "Qwen2ForCausalLM"
5
  ],
mergekit_config.yml CHANGED
@@ -1,8 +1,15 @@
1
-
2
-
3
- base_model: bunnycore/Qwen2.5-7B-MixStock-V0.1+bunnycore/Qwen-2.5-7b-rp-lora
4
- dtype: bfloat16
5
- merge_method: passthrough
6
  models:
7
- - model: bunnycore/Qwen2.5-7B-MixStock-V0.1+bunnycore/Qwen-2.5-7b-rp-lora
8
- tokenizer_source: bunnycore/Qwen2.5-7B-MixStock-V0.1
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  models:
2
+ # Pivot model
3
+ - model: bunnycore/Qwen2.5-7B-RRP-1M
4
+ # Target models
5
+ - model: open-thoughts/OpenThinker-7B
6
+ - model: open-r1/OpenR1-Qwen-7B
7
+ - model: bunnycore/Qwen2.5-7B-MixStock-V0.1
8
+ - model: nvidia/AceInstruct-7B
9
+ merge_method: sce
10
+ base_model: bunnycore/Qwen2.5-7B-RRP-1M
11
+ parameters:
12
+ select_topk: 0.65
13
+ int8_mask: true
14
+ dtype: bfloat16
15
+
model-00001-of-00004.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:6f34dd956765ecc73ad7a9c6e0e573b0324586e1b65a922ba5cb32efef4b2ba4
3
  size 4970978712
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8c2127c19dc89c055407464e036b94d59fc501f510f54b870c8eeb7af4603a1e
3
  size 4970978712
model-00002-of-00004.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:14a3be136f3bd474fe77b6b90a825e1910d247ecdd9cdd65b1fa85f983f2cce1
3
  size 4932751032
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:299b889e81aaf7adb0cc6a95f35d4b73c8b0a742778124cde8004e2623da3d71
3
  size 4932751032
model-00003-of-00004.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:5f68b371c26931d3ec8f1a46f98032efde0e8ee61ce20eeb3ac55ab1827bbe2c
3
  size 4991495808
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:38c87fd618dbc84bdb83152f44d5a758f64669ff2332385ed5a0f58e6c5e49cf
3
  size 4991495808
model-00004-of-00004.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:cf531c55de122b931de87a8450514e3f80e1b42adf0b86dc341dec2de6102ddc
3
  size 330326240
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0ab21de1801cf249039b4f14764a6b81f8e4a34cb146f14f073bb3802b4d81a0
3
  size 330326240