Hugging Face's logo

brittlewis12
/
stablelm-zephyr-3b-GGUF

Text Generation
GGUF
English
causal-lm
stablelm_epoch
conversational
Model card Files Community
stablelm-zephyr-3b-GGUF
File size: 4,148 Bytes 
bb756e3
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
 
---
base_model: stabilityai/stablelm-zephyr-3b
datasets:
  - HuggingFaceH4/ultrachat_200k
  - HuggingFaceH4/ultrafeedback_binarized
  - meta-math/MetaMathQA
  - WizardLM/WizardLM_evol_instruct_V2_196k
  - Intel/orca_dpo_pairs
license: other
license_link: https://huggingface.co/stabilityai/stablelm-zephyr-3b/blob/main/LICENSE
language:
- en
model_creator: stabilityai
model_name: stablelm-zephyr-3b
model_type: stablelm_epoch
inference: false
tags:
- causal-lm
- stablelm_epoch
pipeline_tag: text-generation
prompt_template: |
  <|system|>
  {{system_message}}<|endoftext|>
  <|user|>
  {{prompt}}<|endoftext|>
  <|assistant|>
  
quantized_by: brittlewis12
---

# StableLM Zephyr 3B GGUF

Original model: [StableLM Zephyr 3B](https://huggingface.co/stabilityai/stablelm-zephyr-3b)
Model creator: [Stability AI](https://huggingface.co/stabilityai)

This repo contains GGUF format model files for Stability AI’s StableLM Zephyr 3B.

> StableLM Zephyr 3B is a 3 billion parameter instruction tuned inspired by [HugginFaceH4's Zephyr 7B](https://huggingface.co/HuggingFaceH4/zephyr-7b-beta) training pipeline this model was trained on a mix of publicly available datasets, synthetic datasets using [Direct Preference Optimization (DPO)](https://arxiv.org/abs/2305.18290), evaluation for this model based on [MT Bench](https://tatsu-lab.github.io/alpaca_eval/) and [Alpaca Benchmark](https://tatsu-lab.github.io/alpaca_eval/).


### What is GGUF?

GGUF is a file format for representing AI models. It is the third version of the format, introduced by the llama.cpp team on August 21st 2023. It is a replacement for GGML, which is no longer supported by llama.cpp.
Converted using llama.cpp b1960 ([26d6076](https://github.com/ggerganov/llama.cpp/commits/26d607608d794efa56df3bdb6043a2f94c1d632c))

### Prompt template: Zephyr

```
<|system|>
{{system_message}}<|endoftext|>
<|user|>
{{prompt}}<|endoftext|>
<|assistant|>
```

---

## Download & run with [cnvrs](https://twitter.com/cnvrsai) on iPhone, iPad, and Mac!

![cnvrs.ai](https://pbs.twimg.com/profile_images/1744049151241797632/0mIP-P9e_400x400.jpg)

[cnvrs](https://testflight.apple.com/join/sFWReS7K) is the best app for private, local AI on your device:
- create & save **Characters** with custom system prompts & temperature settings
- download and experiment with any **GGUF model** you can [find on HuggingFace](https://huggingface.co/models?library=gguf)!
- make it your own with custom **Theme colors**
- powered by Metal ⚡️ & [Llama.cpp](https://github.com/ggerganov/llama.cpp), with **haptics** during response streaming!
- **try it out** yourself today, on [Testflight](https://testflight.apple.com/join/sFWReS7K)!
- follow [cnvrs on twitter](https://twitter.com/cnvrsai) to stay up to date

---

## Original Model Evaluations:

![mt-bench](https://cdn-uploads.huggingface.co/production/uploads/6310474ca119d49bc1eb0d80/8WIZS6dAlu5kSH-382pMl.png)

| Model | Size | Alignment | MT-Bench (score) | AlpacaEval (win rate %) |
|-------------|-----|----|---------------|--------------|
| **StableLM Zephyr 3B** 🪁 | 3B | DPO | 6.64 | 76.00 |
| StableLM Zephyr (SFT only) | 3B | SFT | 6.04 | 71.15 | 
| Capybara v1.9 | 3B | dSFT | 5.94 | - |
| MPT-Chat |  7B |dSFT |5.42| -|
| Xwin-LM v0.1 | 7B| dPPO| 6.19| 87.83|
| Mistral-Instruct v0.1 | 7B|  - | 6.84 |-|
| Zephyr-7b-α |7B|  dDPO| 6.88| -|
| Zephyr-7b-β| 7B | dDPO | 7.34 | 90.60 |
| Falcon-Instruct |  40B |dSFT |5.17 |45.71|
| Guanaco | 65B |  SFT |6.41| 71.80|
| Llama2-Chat |  70B |RLHF |6.86| 92.66|
| Vicuna v1.3 |  33B |dSFT |7.12 |88.99|
| WizardLM v1.0 |  70B |dSFT |7.71 |-|
| Xwin-LM v0.1 |   70B |dPPO |- |95.57|
| GPT-3.5-turbo | - |RLHF |7.94 |89.37|
| Claude 2 |  - |RLHF |8.06| 91.36|
| GPT-4 |  -| RLHF |8.99| 95.28|

| Task                | Value                     |
|-----------------------|---------------------------|
| ARC (25-shot)         |  47.0       |
| HellaSwag (10-shot)   | 74.2    |
| MMLU (5-shot)        |   46.3     |
| TruthfulQA (0-shot)   |   46.5 |
| Winogrande (5-shot)   |   65.5 |
| GSM8K (5-shot)        | 42.3        |
| BigBench (Avg) | 35.26 |
| AGI Benchmark (Avg) | 33.23 |