metadata
datasets:
- bigscience/xP3mt
- mc4
license: apache-2.0
language:
- af
- am
- ar
- az
- be
- bg
- bn
- ca
- ceb
- co
- cs
- cy
- da
- de
- el
- en
- eo
- es
- et
- eu
- fa
- fi
- fil
- fr
- fy
- ga
- gd
- gl
- gu
- ha
- haw
- hi
- hmn
- ht
- hu
- hy
- ig
- is
- it
- iw
- ja
- jv
- ka
- kk
- km
- kn
- ko
- ku
- ky
- la
- lb
- lo
- lt
- lv
- mg
- mi
- mk
- ml
- mn
- mr
- ms
- mt
- my
- ne
- nl
- 'no'
- ny
- pa
- pl
- ps
- pt
- ro
- ru
- sd
- si
- sk
- sl
- sm
- sn
- so
- sq
- sr
- st
- su
- sv
- sw
- ta
- te
- tg
- th
- tr
- uk
- und
- ur
- uz
- vi
- xh
- yi
- yo
- zh
- zu
tags:
- text2text-generation
- llama-cpp
- gguf-my-repo
widget:
- text: Life is beautiful! Translate to Mongolian.
example_title: mn-en translation
- text: Le mot japonais «憂鬱» veut dire quoi en Odia?
example_title: jp-or-fr translation
- text: >-
Stell mir eine schwierige Quiz Frage bei der es um Astronomie geht. Bitte
stell die Frage auf Norwegisch.
example_title: de-nb quiz
- text: >-
一个传奇的开端，一个不灭的神话，这不仅仅是一部电影，而是作为一个走进新时代的标签，永远彪炳史册。Would you rate the
previous review as positive, neutral or negative?
example_title: zh-en sentiment
- text: 一个传奇的开端，一个不灭的神话，这不仅仅是一部电影，而是作为一个走进新时代的标签，永远彪炳史册。你认为这句话的立场是赞扬、中立还是批评？
example_title: zh-zh sentiment
- text: Suggest at least five related search terms to "Mạng neural nhân tạo".
example_title: vi-en query
- text: >-
Proposez au moins cinq mots clés concernant «Réseau de neurones
artificiels».
example_title: fr-fr query
- text: >-
Explain in a sentence in Telugu what is backpropagation in neural
networks.
example_title: te-en qa
- text: Why is the sky blue?
example_title: en-en qa
- text: >-
Write a fairy tale about a troll saving a princess from a dangerous
dragon. The fairy tale is a masterpiece that has achieved praise worldwide
and its moral is "Heroes Come in All Shapes and Sizes". Story (in
Spanish):
example_title: es-en fable
- text: >-
Write a fable about wood elves living in a forest that is suddenly invaded
by ogres. The fable is a masterpiece that has achieved praise worldwide
and its moral is "Violence is the last refuge of the incompetent". Fable
(in Hindi):
example_title: hi-en fable
pipeline_tag: text2text-generation
base_model: bigscience/mt0-xxl-mt
model-index:
- name: mt0-xxl-mt
results:
- task:
type: Coreference resolution
dataset:
name: Winogrande XL (xl)
type: winogrande
config: xl
split: validation
revision: a80f460359d1e9a67c006011c94de42a8759430c
metrics:
- type: Accuracy
value: 62.67
- task:
type: Coreference resolution
dataset:
name: XWinograd (en)
type: Muennighoff/xwinograd
config: en
split: test
revision: 9dd5ea5505fad86b7bedad667955577815300cee
metrics:
- type: Accuracy
value: 83.31
- task:
type: Coreference resolution
dataset:
name: XWinograd (fr)
type: Muennighoff/xwinograd
config: fr
split: test
revision: 9dd5ea5505fad86b7bedad667955577815300cee
metrics:
- type: Accuracy
value: 78.31
- task:
type: Coreference resolution
dataset:
name: XWinograd (jp)
type: Muennighoff/xwinograd
config: jp
split: test
revision: 9dd5ea5505fad86b7bedad667955577815300cee
metrics:
- type: Accuracy
value: 80.19
- task:
type: Coreference resolution
dataset:
name: XWinograd (pt)
type: Muennighoff/xwinograd
config: pt
split: test
revision: 9dd5ea5505fad86b7bedad667955577815300cee
metrics:
- type: Accuracy
value: 80.99
- task:
type: Coreference resolution
dataset:
name: XWinograd (ru)
type: Muennighoff/xwinograd
config: ru
split: test
revision: 9dd5ea5505fad86b7bedad667955577815300cee
metrics:
- type: Accuracy
value: 79.05
- task:
type: Coreference resolution
dataset:
name: XWinograd (zh)
type: Muennighoff/xwinograd
config: zh
split: test
revision: 9dd5ea5505fad86b7bedad667955577815300cee
metrics:
- type: Accuracy
value: 82.34
- task:
type: Natural language inference
dataset:
name: ANLI (r1)
type: anli
config: r1
split: validation
revision: 9dbd830a06fea8b1c49d6e5ef2004a08d9f45094
metrics:
- type: Accuracy
value: 49.5
- task:
type: Natural language inference
dataset:
name: ANLI (r2)
type: anli
config: r2
split: validation
revision: 9dbd830a06fea8b1c49d6e5ef2004a08d9f45094
metrics:
- type: Accuracy
value: 42
- task:
type: Natural language inference
dataset:
name: ANLI (r3)
type: anli
config: r3
split: validation
revision: 9dbd830a06fea8b1c49d6e5ef2004a08d9f45094
metrics:
- type: Accuracy
value: 48.17
- task:
type: Natural language inference
dataset:
name: SuperGLUE (cb)
type: super_glue
config: cb
split: validation
revision: 9e12063561e7e6c79099feb6d5a493142584e9e2
metrics:
- type: Accuracy
value: 87.5
- task:
type: Natural language inference
dataset:
name: SuperGLUE (rte)
type: super_glue
config: rte
split: validation
revision: 9e12063561e7e6c79099feb6d5a493142584e9e2
metrics:
- type: Accuracy
value: 84.84
- task:
type: Natural language inference
dataset:
name: XNLI (ar)
type: xnli
config: ar
split: validation
revision: a5a45e4ff92d5d3f34de70aaf4b72c3bdf9f7f16
metrics:
- type: Accuracy
value: 58.03
- task:
type: Natural language inference
dataset:
name: XNLI (bg)
type: xnli
config: bg
split: validation
revision: a5a45e4ff92d5d3f34de70aaf4b72c3bdf9f7f16
metrics:
- type: Accuracy
value: 59.92
- task:
type: Natural language inference
dataset:
name: XNLI (de)
type: xnli
config: de
split: validation
revision: a5a45e4ff92d5d3f34de70aaf4b72c3bdf9f7f16
metrics:
- type: Accuracy
value: 60.16
- task:
type: Natural language inference
dataset:
name: XNLI (el)
type: xnli
config: el
split: validation
revision: a5a45e4ff92d5d3f34de70aaf4b72c3bdf9f7f16
metrics:
- type: Accuracy
value: 59.2
- task:
type: Natural language inference
dataset:
name: XNLI (en)
type: xnli
config: en
split: validation
revision: a5a45e4ff92d5d3f34de70aaf4b72c3bdf9f7f16
metrics:
- type: Accuracy
value: 62.25
- task:
type: Natural language inference
dataset:
name: XNLI (es)
type: xnli
config: es
split: validation
revision: a5a45e4ff92d5d3f34de70aaf4b72c3bdf9f7f16
metrics:
- type: Accuracy
value: 60.92
- task:
type: Natural language inference
dataset:
name: XNLI (fr)
type: xnli
config: fr
split: validation
revision: a5a45e4ff92d5d3f34de70aaf4b72c3bdf9f7f16
metrics:
- type: Accuracy
value: 59.88
- task:
type: Natural language inference
dataset:
name: XNLI (hi)
type: xnli
config: hi
split: validation
revision: a5a45e4ff92d5d3f34de70aaf4b72c3bdf9f7f16
metrics:
- type: Accuracy
value: 57.47
- task:
type: Natural language inference
dataset:
name: XNLI (ru)
type: xnli
config: ru
split: validation
revision: a5a45e4ff92d5d3f34de70aaf4b72c3bdf9f7f16
metrics:
- type: Accuracy
value: 58.67
- task:
type: Natural language inference
dataset:
name: XNLI (sw)
type: xnli
config: sw
split: validation
revision: a5a45e4ff92d5d3f34de70aaf4b72c3bdf9f7f16
metrics:
- type: Accuracy
value: 56.79
- task:
type: Natural language inference
dataset:
name: XNLI (th)
type: xnli
config: th
split: validation
revision: a5a45e4ff92d5d3f34de70aaf4b72c3bdf9f7f16
metrics:
- type: Accuracy
value: 58.03
- task:
type: Natural language inference
dataset:
name: XNLI (tr)
type: xnli
config: tr
split: validation
revision: a5a45e4ff92d5d3f34de70aaf4b72c3bdf9f7f16
metrics:
- type: Accuracy
value: 57.67
- task:
type: Natural language inference
dataset:
name: XNLI (ur)
type: xnli
config: ur
split: validation
revision: a5a45e4ff92d5d3f34de70aaf4b72c3bdf9f7f16
metrics:
- type: Accuracy
value: 55.98
- task:
type: Natural language inference
dataset:
name: XNLI (vi)
type: xnli
config: vi
split: validation
revision: a5a45e4ff92d5d3f34de70aaf4b72c3bdf9f7f16
metrics:
- type: Accuracy
value: 58.92
- task:
type: Natural language inference
dataset:
name: XNLI (zh)
type: xnli
config: zh
split: validation
revision: a5a45e4ff92d5d3f34de70aaf4b72c3bdf9f7f16
metrics:
- type: Accuracy
value: 58.71
- task:
type: Sentence completion
dataset:
name: StoryCloze (2016)
type: story_cloze
config: '2016'
split: validation
revision: e724c6f8cdf7c7a2fb229d862226e15b023ee4db
metrics:
- type: Accuracy
value: 94.66
- task:
type: Sentence completion
dataset:
name: SuperGLUE (copa)
type: super_glue
config: copa
split: validation
revision: 9e12063561e7e6c79099feb6d5a493142584e9e2
metrics:
- type: Accuracy
value: 88
- task:
type: Sentence completion
dataset:
name: XCOPA (et)
type: xcopa
config: et
split: validation
revision: 37f73c60fb123111fa5af5f9b705d0b3747fd187
metrics:
- type: Accuracy
value: 81
- task:
type: Sentence completion
dataset:
name: XCOPA (ht)
type: xcopa
config: ht
split: validation
revision: 37f73c60fb123111fa5af5f9b705d0b3747fd187
metrics:
- type: Accuracy
value: 79
- task:
type: Sentence completion
dataset:
name: XCOPA (id)
type: xcopa
config: id
split: validation
revision: 37f73c60fb123111fa5af5f9b705d0b3747fd187
metrics:
- type: Accuracy
value: 90
- task:
type: Sentence completion
dataset:
name: XCOPA (it)
type: xcopa
config: it
split: validation
revision: 37f73c60fb123111fa5af5f9b705d0b3747fd187
metrics:
- type: Accuracy
value: 88
- task:
type: Sentence completion
dataset:
name: XCOPA (qu)
type: xcopa
config: qu
split: validation
revision: 37f73c60fb123111fa5af5f9b705d0b3747fd187
metrics:
- type: Accuracy
value: 56
- task:
type: Sentence completion
dataset:
name: XCOPA (sw)
type: xcopa
config: sw
split: validation
revision: 37f73c60fb123111fa5af5f9b705d0b3747fd187
metrics:
- type: Accuracy
value: 81
- task:
type: Sentence completion
dataset:
name: XCOPA (ta)
type: xcopa
config: ta
split: validation
revision: 37f73c60fb123111fa5af5f9b705d0b3747fd187
metrics:
- type: Accuracy
value: 81
- task:
type: Sentence completion
dataset:
name: XCOPA (th)
type: xcopa
config: th
split: validation
revision: 37f73c60fb123111fa5af5f9b705d0b3747fd187
metrics:
- type: Accuracy
value: 76
- task:
type: Sentence completion
dataset:
name: XCOPA (tr)
type: xcopa
config: tr
split: validation
revision: 37f73c60fb123111fa5af5f9b705d0b3747fd187
metrics:
- type: Accuracy
value: 76
- task:
type: Sentence completion
dataset:
name: XCOPA (vi)
type: xcopa
config: vi
split: validation
revision: 37f73c60fb123111fa5af5f9b705d0b3747fd187
metrics:
- type: Accuracy
value: 85
- task:
type: Sentence completion
dataset:
name: XCOPA (zh)
type: xcopa
config: zh
split: validation
revision: 37f73c60fb123111fa5af5f9b705d0b3747fd187
metrics:
- type: Accuracy
value: 87
- task:
type: Sentence completion
dataset:
name: XStoryCloze (ar)
type: Muennighoff/xstory_cloze
config: ar
split: validation
revision: 8bb76e594b68147f1a430e86829d07189622b90d
metrics:
- type: Accuracy
value: 91
- task:
type: Sentence completion
dataset:
name: XStoryCloze (es)
type: Muennighoff/xstory_cloze
config: es
split: validation
revision: 8bb76e594b68147f1a430e86829d07189622b90d
metrics:
- type: Accuracy
value: 93.38
- task:
type: Sentence completion
dataset:
name: XStoryCloze (eu)
type: Muennighoff/xstory_cloze
config: eu
split: validation
revision: 8bb76e594b68147f1a430e86829d07189622b90d
metrics:
- type: Accuracy
value: 91.13
- task:
type: Sentence completion
dataset:
name: XStoryCloze (hi)
type: Muennighoff/xstory_cloze
config: hi
split: validation
revision: 8bb76e594b68147f1a430e86829d07189622b90d
metrics:
- type: Accuracy
value: 90.73
- task:
type: Sentence completion
dataset:
name: XStoryCloze (id)
type: Muennighoff/xstory_cloze
config: id
split: validation
revision: 8bb76e594b68147f1a430e86829d07189622b90d
metrics:
- type: Accuracy
value: 93.05
- task:
type: Sentence completion
dataset:
name: XStoryCloze (my)
type: Muennighoff/xstory_cloze
config: my
split: validation
revision: 8bb76e594b68147f1a430e86829d07189622b90d
metrics:
- type: Accuracy
value: 86.7
- task:
type: Sentence completion
dataset:
name: XStoryCloze (ru)
type: Muennighoff/xstory_cloze
config: ru
split: validation
revision: 8bb76e594b68147f1a430e86829d07189622b90d
metrics:
- type: Accuracy
value: 91.66
- task:
type: Sentence completion
dataset:
name: XStoryCloze (sw)
type: Muennighoff/xstory_cloze
config: sw
split: validation
revision: 8bb76e594b68147f1a430e86829d07189622b90d
metrics:
- type: Accuracy
value: 89.61
- task:
type: Sentence completion
dataset:
name: XStoryCloze (te)
type: Muennighoff/xstory_cloze
config: te
split: validation
revision: 8bb76e594b68147f1a430e86829d07189622b90d
metrics:
- type: Accuracy
value: 90.4
- task:
type: Sentence completion
dataset:
name: XStoryCloze (zh)
type: Muennighoff/xstory_cloze
config: zh
split: validation
revision: 8bb76e594b68147f1a430e86829d07189622b90d
metrics:
- type: Accuracy
value: 93.05
Markobes/mt0-xxl-mt-Q5_K_M-GGUF
This model was converted to GGUF format from bigscience/mt0-xxl-mt using llama.cpp via the ggml.ai's GGUF-my-repo space.
Refer to the original model card for more details on the model.
Use with llama.cpp
Install llama.cpp through brew (works on Mac and Linux)
brew install llama.cpp
Invoke the llama.cpp server or the CLI.
CLI:
llama-cli --hf-repo Markobes/mt0-xxl-mt-Q5_K_M-GGUF --hf-file mt0-xxl-mt-q5_k_m.gguf -p "The meaning to life and the universe is"
Server:
llama-server --hf-repo Markobes/mt0-xxl-mt-Q5_K_M-GGUF --hf-file mt0-xxl-mt-q5_k_m.gguf -c 2048
Note: You can also use this checkpoint directly through the usage steps listed in the Llama.cpp repo as well.
Step 1: Clone llama.cpp from GitHub.
git clone https://github.com/ggerganov/llama.cpp
Step 2: Move into the llama.cpp folder and build it with LLAMA_CURL=1 flag along with other hardware-specific flags (for ex: LLAMA_CUDA=1 for Nvidia GPUs on Linux).
cd llama.cpp && LLAMA_CURL=1 make
Step 3: Run inference through the main binary.
./llama-cli --hf-repo Markobes/mt0-xxl-mt-Q5_K_M-GGUF --hf-file mt0-xxl-mt-q5_k_m.gguf -p "The meaning to life and the universe is"
or
./llama-server --hf-repo Markobes/mt0-xxl-mt-Q5_K_M-GGUF --hf-file mt0-xxl-mt-q5_k_m.gguf -c 2048