|
--- |
|
license: apache-2.0 |
|
tags: |
|
- merge |
|
- mergekit |
|
- ShinojiResearch/Senku-70B-Full |
|
- 152334H/miqu-1-70b-sf |
|
--- |
|
|
|
# ECE-TW3-JRGL-V1 |
|
|
|
## This model has been produced by : |
|
|
|
- [Louis Garcia](https://www.linkedin.com/in/louis-garcia-profil/), engineering student at [French Engineering School ECE](https://www.ece.fr/en/) |
|
- [Matthieu Jollard](https://www.linkedin.com/in/matthieu-jollard/), engineering student at [French Engineering School ECE](https://www.ece.fr/en/) |
|
|
|
## Under the supervision of : |
|
|
|
- [Andre-Louis Rochet](https://www.linkedin.com/in/andrelouisrochet/), Lecturer at ECE & Co-Founder of [TW3 Partners](https://tw3partners.fr/) |
|
- [Paul Lemaistre](https://www.linkedin.com/in/paul-lemaistre/), CTO of [TW3 Partners](https://tw3partners.fr/) |
|
|
|
## With the contribution of : |
|
|
|
- ECE engineering school as sponsor and financial contributor |
|
- RunPod as financial contributor |
|
|
|
## About ECE |
|
>_**ECE**, a multi-program, multi-campus, and multi-sector engineering school specializing in digital engineering, |
|
> trains engineers and technology experts for the 21st century, capable of meeting the challenges of the dual digital and sustainable development revolutions. |
|
>[French Engineering School ECE](https://www.ece.fr/en/)_ |
|
## Description |
|
|
|
ECE-TW3-JRGL-V1 is a merge of the following models using **[mergekit](https://github.com/cg123/mergekit)**: |
|
|
|
* [ShinojiResearch/Senku-70B-Full](https://huggingface.co/ShinojiResearch/Senku-70B-Full) |
|
* [152334H/miqu-1-70b-sf](https://huggingface.co/152334H/miqu-1-70b-sf) |
|
|
|
```yaml |
|
slices: |
|
- sources: |
|
- model: ShinojiResearch/Senku-70B-Full |
|
layer_range: [0, 80] |
|
- model: 152334H/miqu-1-70b-sf |
|
layer_range: [0, 80] |
|
merge_method: slerp |
|
base_model: 152334H/miqu-1-70b-sf |
|
parameters: |
|
t: |
|
- filter: self_attn |
|
value: [0, 0.5, 0.3, 0.7, 1] |
|
- filter: mlp |
|
value: [1, 0.5, 0.7, 0.3, 0] |
|
- value: 0.5 |
|
dtype: float16 |
|
``` |
|
|
|
## Results |
|
|
|
- ECE-TW3-JRGL-v1 scores 83.07 on [EQ-Bench V2](https://eqbench.com/index.html) |
|
|
|
--- |
|
@misc{paech2023eqbench, |
|
title={EQ-Bench: An Emotional Intelligence Benchmark for Large Language Models}, |
|
author={Samuel J. Paech}, |
|
year={2023}, |
|
eprint={2312.06281}, |
|
archivePrefix={arXiv}, |
|
primaryClass={cs.CL} |
|
} |
|
--- |
|
|
