File size: 4,071 Bytes
8fe2350 afdc737 8fe2350 afdc737 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 |
---
language:
- pt
- en
license: cc
tags:
- text-generation-inference
- transformers
- mistral
- mixtral
- gguf
- brazil
- brasil
- portuguese
---
# BotBot Cabra Mixtral 8x7b
Esse modelo é um finetune do [Mixtral 8x7b](https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1) com o dataset Cabra 30k. Esse modelo é optimizado para português. Ele apresenta melhoria em varios benchmarks brasileiros em comparação com o modelo base.
**Conheça os nossos outros modelos: [Cabra](https://huggingface.co/collections/botbot-ai/models-6604c2069ceef04f834ba99b).**
### dataset: Cabra 30k
Dataset interno para finetuning. Vamos lançar em breve.
### Quantização / GGUF
Colocamos diversas versões (GGUF) quantanizadas no branch "quantanization".
### Exemplo
```
<s> [INST] who is Elon Musk? [/INST]Elon Musk é um empreendedor, inventor e capitalista americano. Ele é o fundador, CEO e CTO da SpaceX, CEO da Neuralink e fundador do The Boring Company. Musk também é o proprietário do Twitter.</s>
```
## Uso
O modelo é destinado, por agora, a fins de pesquisa. As áreas e tarefas de pesquisa possíveis incluem:
- Pesquisa sobre modelos gerativos.
- Investigação e compreensão das limitações e viéses de modelos gerativos.
**Proibido para uso comercial. Somente Pesquisa.**
### Evals
Tasks Version Filter n-shot Metric Value Stderr
assin2_rte 1.1 all 15 f1_macro 0.9095 ± 0.0041
all 15 acc 0.9097 ± 0.0041
assin2_sts 1.1 all 15 pearson 0.7763 ± 0.0068
all 15 mse 0.4610 ± N/A
bluex 1.1 all 3 acc 0.6412 ± 0.0103
exam_id__UNICAMP_2021_2 3 acc 0.5882 ± 0.0397
exam_id__USP_2023 3 acc 0.7045 ± 0.0397
exam_id__UNICAMP_2020 3 acc 0.6545 ± 0.0371
exam_id__UNICAMP_2023 3 acc 0.7442 ± 0.0384
exam_id__UNICAMP_2018 3 acc 0.5926 ± 0.0386
exam_id__UNICAMP_2022 3 acc 0.6154 ± 0.0451
exam_id__USP_2019 3 acc 0.6250 ± 0.0442
exam_id__USP_2021 3 acc 0.6346 ± 0.0387
exam_id__UNICAMP_2021_1 3 acc 0.5000 ± 0.0427
exam_id__USP_2022 3 acc 0.6531 ± 0.0393
exam_id__UNICAMP_2024 3 acc 0.5556 ± 0.0428
exam_id__USP_2024 3 acc 0.8537 ± 0.0319
exam_id__USP_2018 3 acc 0.6296 ± 0.0379
exam_id__USP_2020 3 acc 0.6071 ± 0.0375
exam_id__UNICAMP_2019 3 acc 0.7000 ± 0.0374
enem 1.1 all 3 acc 0.7810 ± 0.0063
exam_id__2013 3 acc 0.7685 ± 0.0236
exam_id__2010 3 acc 0.8205 ± 0.0205
exam_id__2012 3 acc 0.8276 ± 0.0202
exam_id__2016_2 3 acc 0.7886 ± 0.0213
exam_id__2016 3 acc 0.8017 ± 0.0209
exam_id__2022 3 acc 0.6541 ± 0.0238
exam_id__2009 3 acc 0.8087 ± 0.0212
exam_id__2015 3 acc 0.8067 ± 0.0208
exam_id__2017 3 acc 0.7759 ± 0.0223
exam_id__2014 3 acc 0.7798 ± 0.0230
exam_id__2023 3 acc 0.7037 ± 0.0228
exam_id__2011 3 acc 0.8632 ± 0.0183
faquad_nli 1.1 all 15 f1_macro 0.7893 ± 0.0137
all 15 acc 0.8554 ± 0.0097
hatebr_offensive_binary 1.0 all 25 f1_macro 0.7800 ± 0.0080
all 25 acc 0.7879 ± 0.0077
oab_exams 1.5 all 3 acc 0.5549 ± 0.0061
exam_id__2014-14 3 acc 0.6375 ± 0.0311
exam_id__2010-01 3 acc 0.4471 ± 0.0312
exam_id__2016-20a 3 acc 0.5000 ± 0.0323
exam_id__2013-10 3 acc 0.5750 ± 0.0318
exam_id__2010-02 3 acc 0.5500 ± 0.0288
exam_id__2011-04 3 acc 0.5750 ± 0.0319
exam_id__2013-12 3 acc 0.6000 ± 0.0316
exam_id__2017-22 3 acc 0.6000 ± 0.0316
exam_id__2015-16 3 acc 0.5625 ± 0.0320
exam_id__2012-09 3 acc 0.4416 ± 0.0326
exam_id__2016-19 3 acc 0.5641 ± 0.0326
exam_id__2017-23 3 acc 0.5500 ± 0.0321
exam_id__2013-11 3 acc 0.5500 ± 0.0321
exam_id__2012-08 3 acc 0.5500 ± 0.0323
exam_id__2011-03 3 acc 0.5152 ± 0.0290
exam_id__2015-17 3 acc 0.6538 ± 0.0310
exam_id__2015-18 3 acc 0.6125 ± 0.0314
exam_id__2014-15 3 acc 0.6538 ± 0.0310
exam_id__2018-25 3 acc 0.5750 ± 0.0320
exam_id__2012-06a 3 acc 0.5750 ± 0.0318
exam_id__2017-24 3 acc 0.4875 ± 0.0323
exam_id__2011-05 3 acc 0.5250 ± 0.0323
exam_id__2012-06 3 acc 0.5750 ± 0.0319
exam_id__2016-21 3 acc 0.4875 ± 0.0322
exam_id__2012-07 3 acc 0.5000 ± 0.0323
exam_id__2016-20 3 acc 0.5500 ± 0.0322
exam_id__2014-13 3 acc 0.5875 ± 0.0316
portuguese_hate_speech_binary 1.0 all 25 f1_macro 0.6954 ± 0.0114
all 25 acc 0.7086 ± 0.0110
|