metadata
license: apache-2.0
tags:
- merge
- mergekit
- ShinojiResearch/Senku-70B-Full
- 152334H/miqu-1-70b-sf
ECE-TW3-JRGL-V1
This model has been produced by :
- Louis Garcia, engineering student at French Engineering School ECE
- Matthieu Jollard, engineering student at French Engineering School ECE
Under the supervision of :
- Andre-Louis Rochet, Lecturer at ECE & Co-Founder of TW3 Partners
- Paul Lemaistre, CTO of TW3 Partners
With the contribution of :
- ECE engineering school as sponsor and financial contributor
- RunPod as financial contributor
About ECE
ECE, a multi-program, multi-campus, and multi-sector engineering school specializing in digital engineering, trains engineers and technology experts for the 21st century, capable of meeting the challenges of the dual digital and sustainable development revolutions. French Engineering School ECE
Description
ECE-TW3-JRGL-V1 is a merge of the following models using mergekit:
slices:
- sources:
- model: ShinojiResearch/Senku-70B-Full
layer_range: [0, 80]
- model: 152334H/miqu-1-70b-sf
layer_range: [0, 80]
merge_method: slerp
base_model: 152334H/miqu-1-70b-sf
parameters:
t:
- filter: self_attn
value: [0, 0.5, 0.3, 0.7, 1]
- filter: mlp
value: [1, 0.5, 0.7, 0.3, 0]
- value: 0.5
dtype: float16
Results
- ECE-TW3-JRGL-v1 scores 83.07 on EQ-Bench V2