Hugging Face's logo

spmurrayzzz
/
Mistral-Syndicate-7B

Text Generation
Transformers
PyTorch
Safetensors
English
mistral
mistral-7b
instruct
finetune
synthetic data
distillation
text-generation-inference
Inference Endpoints
Model card Files Community
3
Mistral-Syndicate-7B
File size: 2,293 Bytes 
3f2c168
ae74855
 
 
226bb45
 
 
 
 
 
ae74855
226bb45
 
 
3f2c168
e71fc80
226bb45
 
e71fc80
 
 
d95d34d
 
 
 
 
 
 
f890203
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
6780f05
 
 
 
 
 
 
 
 
 
 
ae74855
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
 
---
language:
- en
license: apache-2.0
tags:
- mistral-7b
- instruct
- finetune
- synthetic data
- distillation
base_model: mistralai/Mistral-7B-v0.1
model-index:
- name: Mistral-Syndicate-7B
  results: []
---

## Mistral-Syndicate-7B


<img src="https://cdn-uploads.huggingface.co/production/uploads/64712903631bb7660aaa7183/yMzb5tbgygtlHnoyR37YO.png" width="600px" />

## Model Description:

Mistral Syndicate is in no way a state-of-the-art model, rather it is a fine-tuning experiment to explore
the training dynamics specific to large language models. The dataset used in finetuning was generated via
a "syndicate" of other open language models both of similar parameter size and larger. Each model would generate a
response for a given instruction, and the group would vote on which model's response was best.

The instruction inputs used for the output label synthesis were a curated subset of [VMWare/open-instruct](https://huggingface.co/datasets/VMware/open-instruct)
with additional instructions synthesized from scratch.

## Prompt template

With context
```
Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.

### Instruction:

### Input:

### Response:
```

Without context
```
Below is an instruction that describes a task. Write a response that appropriately completes the request.

### Instruction:

### Response:
```

## Evaluation Results
_12.30.23_
| Benchmark  | Result |
|------------|--------|
| ARC        | 60.84  |
| HellaSwag  | 82.91  |
| MMLU       | 60.83  |
| TruthfulQA | 43.71  |
| Winogrande | 78.61  |
| GSM8K      | 44.50  |


# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_spmurrayzzz__Mistral-Syndicate-7B)

|             Metric              |Value|
|---------------------------------|----:|
|Avg.                             |61.90|
|AI2 Reasoning Challenge (25-Shot)|60.84|
|HellaSwag (10-Shot)              |82.91|
|MMLU (5-Shot)                    |60.83|
|TruthfulQA (0-shot)              |43.71|
|Winogrande (5-shot)              |78.61|
|GSM8k (5-shot)                   |44.50|