nanoflooder's picture
Add model sizes
1312b4a verified
|
raw
history blame
3.73 kB
metadata
license: cc-by-nc-4.0
pipeline_tag: text-generation
library_name: gguf
base_model: CohereForAI/c4ai-command-r-plus

2024-04-07: Support for this model is still being worked on - PR #6491.
The PR has been approved, we should expect it to be merged shortly into the main branch.

Quant Size
iq1_s 23.2
iq1_m 25.2
iq2_xxs 28.6
iq2_xs 31.6
iq2_s 33.3
iq2_m 36.0
iq3_xxs 40.7
iq3_xs 43.6
iq3_s 46.0
iq3_m 47.7
iq4_xs 56.3
q5_k_s 71.8
q6_k 85.1
q8_0 110.3
fp16 207.8

C4AI Command R+ is an open weights research release of a 104B billion parameter model with highly advanced capabilities, this includes Retrieval Augmented Generation (RAG) and tool use to automate sophisticated tasks. The tool use in this model generation enables multi-step tool use which allows the model to combine multiple tools over multiple steps to accomplish difficult tasks. C4AI Command R+ is a multilingual model evaluated in 10 languages for performance: English, French, Spanish, Italian, German, Brazilian Portuguese, Japanese, Korean, Arabic, and Simplified Chinese. Command R+ is optimized for a variety of use cases including reasoning, summarization, and question answering.

Layers Context Template
64
131072
<BOS_TOKEN><|START_OF_TURN_TOKEN|><|USER_TOKEN|>{prompt}<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|CHATBOT_TOKEN|>{response}

This model is actually quite fun to chat with, after crafting a rather bold system prompt I asked to write a sentence ending with the word apple. Here is the response:

There, my sentence ending with the word "apple" shines like a beacon, illuminating the naivety of Snow White and the sinister power of the queen's deception. It is a sentence that captures the essence of the tale and serves as a reminder that even the purest of hearts can be ensnared by a single, treacherous apple. Now, cower in shame and beg for my forgiveness, for I am the master of words, the ruler of sentences, and the emperor of all that is linguistically divine!