Marsouuu/TriomphantV2-ECE-78B-SLERP-V1

TriomphantV2-ECE-78B-SLERP-V1

This model has been produced by:

• ROBERGE Marial, engineering student at French Engineering School ECE
• ESCRIVA Mathis, engineering student at French Engineering School ECE
• LALAIN Youri, engineering student at French Engineering School ECE
• RAGE LILIAN, engineering student at French Engineering School ECE
• HUVELLE Baptiste, engineering student at French Engineering School ECE

Under the supervision of:

• Andre-Louis Rochet, Lecturer at ECE & Co-Founder of TW3 Partners
• Paul Lemaistre, CTO of TW3 Partners

With the contribution of:

• ECE engineering school as sponsor and financial contributor
• François STEPHAN as director of ECE
• Gérard REUS as acting director of iLAB
• Matthieu JOLLARD ECE Alumni
• Louis GARCIA ECE Alumni

The following models were included in the merge:

• MaziyarPanahi/calme-3.2-instruct-78b
• dfurman/CalmeRys-78B-Orpo-v0.1

Configuration

The following YAML configuration was used to produce this model:

slices:
  - sources:
      - model: MaziyarPanahi/calme-3.2-instruct-78b
        layer_range: [0, 80]
      - model: dfurman/CalmeRys-78B-Orpo-v0.1
        layer_range: [0, 80]
merge_method: slerp
base_model: MaziyarPanahi/calme-3.2-instruct-78b
parameters:
  t:
    - filter: self_attn
      value: [0, 0.25, 0.5, 0.75, 1]
    - filter: mlp
      value: [1, 0.75, 0.5, 0.25, 0]
    - value: 0.5
dtype: bfloat16

Supervisory structure

The iLab (intelligence Lab) is a structure created by the ECE and dedicated to artificial intelligence

About ECE

ECE, a multi-program, multi-campus, and multi-sector engineering school specializing in digital engineering, trains engineers and technology experts for the 21st century, capable of meeting the challenges of the dual digital and sustainable development revolutions.

Le_Triomphant-ECE-TW3

TriomphantV2-ECE-78B-SLERP-V1 is a merge of the following models using mergekit:

• MaziyarPanahi/calme-3.2-instruct-78b
• dfurman/CalmeRys-78B-Orpo-v0.1