File size: 2,222 Bytes
68112e1 2f08c7a 58e545c 2f08c7a 58e545c 2f08c7a 58e545c 68112e1 f1916d0 58e545c |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 |
---
license: apache-2.0
tags:
- merge
- mergekit
- ShinojiResearch/Senku-70B-Full
- 152334H/miqu-1-70b-sf
---
# ECE-TW3-JRGL-V1
## This model has been produced by :
- [Louis Garcia](https://www.linkedin.com/in/louis-garcia-profil/), engineering student at [French Engineering School ECE](https://www.ece.fr/en/)
- [Matthieu Jollard](https://www.linkedin.com/in/matthieu-jollard/), engineering student at [French Engineering School ECE](https://www.ece.fr/en/)
## Under the supervision of :
- [Andre-Louis Rochet](https://www.linkedin.com/in/andrelouisrochet/), Lecturer at ECE & Co-Founder of [TW3 Partners](https://tw3partners.fr/)
- [Paul Lemaistre](https://www.linkedin.com/in/paul-lemaistre/), CTO of [TW3 Partners](https://tw3partners.fr/)
## With the contribution of :
- ECE engineering school as sponsor and financial contributor
- RunPod as financial contributor
## About ECE
>_**ECE**, a multi-program, multi-campus, and multi-sector engineering school specializing in digital engineering,
> trains engineers and technology experts for the 21st century, capable of meeting the challenges of the dual digital and sustainable development revolutions.
>[French Engineering School ECE](https://www.ece.fr/en/)_
## Description
ECE-TW3-JRGL-V1 is a merge of the following models using **[mergekit](https://github.com/cg123/mergekit)**:
* [ShinojiResearch/Senku-70B-Full](https://huggingface.co/ShinojiResearch/Senku-70B-Full)
* [152334H/miqu-1-70b-sf](https://huggingface.co/152334H/miqu-1-70b-sf)
```yaml
slices:
- sources:
- model: ShinojiResearch/Senku-70B-Full
layer_range: [0, 80]
- model: 152334H/miqu-1-70b-sf
layer_range: [0, 80]
merge_method: slerp
base_model: 152334H/miqu-1-70b-sf
parameters:
t:
- filter: self_attn
value: [0, 0.5, 0.3, 0.7, 1]
- filter: mlp
value: [1, 0.5, 0.7, 0.3, 0]
- value: 0.5
dtype: float16
```
## Results
- ECE-TW3-JRGL-v1 scores 83.07 on [EQ-Bench V2](https://eqbench.com/index.html)
---
@misc{paech2023eqbench,
title={EQ-Bench: An Emotional Intelligence Benchmark for Large Language Models},
author={Samuel J. Paech},
year={2023},
eprint={2312.06281},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
---
|