File size: 8,406 Bytes
4aadafb
 
6ef6bad
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
a2e871b
 
 
4aadafb
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
059a39e
 
 
 
 
4aadafb
 
60c55b4
4aadafb
 
 
 
 
 
 
 
 
 
 
 
60c55b4
 
4aadafb
60c55b4
 
 
 
 
4aadafb
60c55b4
 
 
 
4aadafb
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
---
base_model: CohereForAI/aya-23-35B
extra_gated_fields:
  Affiliation: text
  Country:
    options:
    - Aruba
    - Afghanistan
    - Angola
    - Anguilla
    - Åland Islands
    - Albania
    - Andorra
    - United Arab Emirates
    - Argentina
    - Armenia
    - American Samoa
    - Antarctica
    - French Southern Territories
    - Antigua and Barbuda
    - Australia
    - Austria
    - Azerbaijan
    - Burundi
    - Belgium
    - Benin
    - Bonaire Sint Eustatius and Saba
    - Burkina Faso
    - Bangladesh
    - Bulgaria
    - Bahrain
    - Bahamas
    - Bosnia and Herzegovina
    - Saint Barthélemy
    - Belarus
    - Belize
    - Bermuda
    - Plurinational State of Bolivia
    - Brazil
    - Barbados
    - Brunei-Darussalam
    - Bhutan
    - Bouvet-Island
    - Botswana
    - Central African Republic
    - Canada
    - Cocos (Keeling) Islands
    - Switzerland
    - Chile
    - China
    - Côte-dIvoire
    - Cameroon
    - Democratic Republic of the Congo
    - Cook Islands
    - Colombia
    - Comoros
    - Cabo Verde
    - Costa Rica
    - Cuba
    - Curaçao
    - Christmas Island
    - Cayman Islands
    - Cyprus
    - Czechia
    - Germany
    - Djibouti
    - Dominica
    - Denmark
    - Dominican Republic
    - Algeria
    - Ecuador
    - Egypt
    - Eritrea
    - Western Sahara
    - Spain
    - Estonia
    - Ethiopia
    - Finland
    - Fiji
    - Falkland Islands (Malvinas)
    - France
    - Faroe Islands
    - Federated States of Micronesia
    - Gabon
    - United Kingdom
    - Georgia
    - Guernsey
    - Ghana
    - Gibraltar
    - Guinea
    - Guadeloupe
    - Gambia
    - Guinea Bissau
    - Equatorial Guinea
    - Greece
    - Grenada
    - Greenland
    - Guatemala
    - French Guiana
    - Guam
    - Guyana
    - Hong Kong
    - Heard Island and McDonald Islands
    - Honduras
    - Croatia
    - Haiti
    - Hungary
    - Indonesia
    - Isle of Man
    - India
    - British Indian Ocean Territory
    - Ireland
    - Islamic Republic of Iran
    - Iraq
    - Iceland
    - Israel
    - Italy
    - Jamaica
    - Jersey
    - Jordan
    - Japan
    - Kazakhstan
    - Kenya
    - Kyrgyzstan
    - Cambodia
    - Kiribati
    - Saint-Kitts-and-Nevis
    - South Korea
    - Kuwait
    - Lao-Peoples-Democratic-Republic
    - Lebanon
    - Liberia
    - Libya
    - Saint-Lucia
    - Liechtenstein
    - Sri Lanka
    - Lesotho
    - Lithuania
    - Luxembourg
    - Latvia
    - Macao
    - Saint Martin (French-part)
    - Morocco
    - Monaco
    - Republic of Moldova
    - Madagascar
    - Maldives
    - Mexico
    - Marshall Islands
    - North Macedonia
    - Mali
    - Malta
    - Myanmar
    - Montenegro
    - Mongolia
    - Northern Mariana Islands
    - Mozambique
    - Mauritania
    - Montserrat
    - Martinique
    - Mauritius
    - Malawi
    - Malaysia
    - Mayotte
    - Namibia
    - New Caledonia
    - Niger
    - Norfolk Island
    - Nigeria
    - Nicaragua
    - Niue
    - Netherlands
    - Norway
    - Nepal
    - Nauru
    - New Zealand
    - Oman
    - Pakistan
    - Panama
    - Pitcairn
    - Peru
    - Philippines
    - Palau
    - Papua New Guinea
    - Poland
    - Puerto Rico
    - North Korea
    - Portugal
    - Paraguay
    - State of Palestine
    - French Polynesia
    - Qatar
    - Réunion
    - Romania
    - Russia
    - Rwanda
    - Saudi Arabia
    - Sudan
    - Senegal
    - Singapore
    - South Georgia and the South Sandwich Islands
    - Saint Helena Ascension and Tristan da Cunha
    - Svalbard and Jan Mayen
    - Solomon Islands
    - Sierra Leone
    - El Salvador
    - San Marino
    - Somalia
    - Saint Pierre and Miquelon
    - Serbia
    - South Sudan
    - Sao Tome and Principe
    - Suriname
    - Slovakia
    - Slovenia
    - Sweden
    - Eswatini
    - Sint Maarten (Dutch-part)
    - Seychelles
    - Syrian Arab Republic
    - Turks and Caicos Islands
    - Chad
    - Togo
    - Thailand
    - Tajikistan
    - Tokelau
    - Turkmenistan
    - Timor Leste
    - Tonga
    - Trinidad and Tobago
    - Tunisia
    - Turkey
    - Tuvalu
    - Taiwan
    - United Republic of Tanzania
    - Uganda
    - Ukraine
    - United States Minor Outlying Islands
    - Uruguay
    - United-States
    - Uzbekistan
    - Holy See (Vatican City State)
    - Saint Vincent and the Grenadines
    - Bolivarian Republic of Venezuela
    - Virgin Islands British
    - Virgin Islands U.S.
    - VietNam
    - Vanuatu
    - Wallis and Futuna
    - Samoa
    - Yemen
    - South Africa
    - Zambia
    - Zimbabwe
    type: select
  I agree to use this model for non-commercial use ONLY: checkbox
  Name: text
extra_gated_prompt: By submitting this form, you agree to the [License Agreement](https://cohere.com/c4ai-cc-by-nc-license)  and
  acknowledge that the information you provide will be collected, used, and shared
  in accordance with Cohere’s [Privacy Policy]( https://cohere.com/privacy). You’ll
  receive email updates about C4AI and Cohere research, events, products and services.
  You can unsubscribe at any time.
language:
- en
- fr
- de
- es
- it
- pt
- ja
- ko
- zh
- ar
- el
- fa
- pl
- id
- cs
- he
- hi
- nl
- ro
- ru
- tr
- uk
- vi
library_name: transformers
license: cc-by-nc-4.0
quantized_by: mradermacher
---
## About

<!-- ### quantize_version: 2 -->
<!-- ### output_tensor_quantised: 1 -->
<!-- ### convert_type: hf -->
<!-- ### vocab_type:  -->
static quants of https://huggingface.co/CohereForAI/aya-23-35B

<!-- provided-files -->
weighted/imatrix quants are available at https://huggingface.co/mradermacher/aya-23-35B-i1-GGUF
## Usage

If you are unsure how to use GGUF files, refer to one of [TheBloke's
READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for
more details, including on how to concatenate multi-part files.

## Provided Quants

(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)

| Link | Type | Size/GB | Notes |
|:-----|:-----|--------:|:------|
| [GGUF](https://huggingface.co/mradermacher/aya-23-35B-GGUF/resolve/main/aya-23-35B.Q2_K.gguf) | Q2_K | 13.9 |  |
| [GGUF](https://huggingface.co/mradermacher/aya-23-35B-GGUF/resolve/main/aya-23-35B.IQ3_XS.gguf) | IQ3_XS | 15.2 |  |
| [GGUF](https://huggingface.co/mradermacher/aya-23-35B-GGUF/resolve/main/aya-23-35B.IQ3_S.gguf) | IQ3_S | 16.0 | beats Q3_K* |
| [GGUF](https://huggingface.co/mradermacher/aya-23-35B-GGUF/resolve/main/aya-23-35B.Q3_K_S.gguf) | Q3_K_S | 16.0 |  |
| [GGUF](https://huggingface.co/mradermacher/aya-23-35B-GGUF/resolve/main/aya-23-35B.IQ3_M.gguf) | IQ3_M | 16.8 |  |
| [GGUF](https://huggingface.co/mradermacher/aya-23-35B-GGUF/resolve/main/aya-23-35B.Q3_K_M.gguf) | Q3_K_M | 17.7 | lower quality |
| [GGUF](https://huggingface.co/mradermacher/aya-23-35B-GGUF/resolve/main/aya-23-35B.Q3_K_L.gguf) | Q3_K_L | 19.2 |  |
| [GGUF](https://huggingface.co/mradermacher/aya-23-35B-GGUF/resolve/main/aya-23-35B.IQ4_XS.gguf) | IQ4_XS | 19.4 |  |
| [GGUF](https://huggingface.co/mradermacher/aya-23-35B-GGUF/resolve/main/aya-23-35B.Q4_K_S.gguf) | Q4_K_S | 20.5 | fast, recommended |
| [GGUF](https://huggingface.co/mradermacher/aya-23-35B-GGUF/resolve/main/aya-23-35B.Q4_K_M.gguf) | Q4_K_M | 21.6 | fast, recommended |
| [GGUF](https://huggingface.co/mradermacher/aya-23-35B-GGUF/resolve/main/aya-23-35B.Q5_K_S.gguf) | Q5_K_S | 24.4 |  |
| [GGUF](https://huggingface.co/mradermacher/aya-23-35B-GGUF/resolve/main/aya-23-35B.Q5_K_M.gguf) | Q5_K_M | 25.1 |  |
| [GGUF](https://huggingface.co/mradermacher/aya-23-35B-GGUF/resolve/main/aya-23-35B.Q6_K.gguf) | Q6_K | 28.8 | very good quality |
| [GGUF](https://huggingface.co/mradermacher/aya-23-35B-GGUF/resolve/main/aya-23-35B.Q8_0.gguf) | Q8_0 | 37.3 | fast, best quality |

Here is a handy graph by ikawrakow comparing some lower-quality quant
types (lower is better):

![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png)

And here are Artefact2's thoughts on the matter:
https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9

## FAQ / Model Request

See https://huggingface.co/mradermacher/model_requests for some answers to
questions you might have and/or if you want some other model quantized.

## Thanks

I thank my company, [nethype GmbH](https://www.nethype.de/), for letting
me use its servers and providing upgrades to my workstation to enable
this work in my free time.

<!-- end -->