File size: 2,884 Bytes
3080521 9d71140 3080521 9d71140 3080521 e5e63c5 abaf30e 3080521 abaf30e 3080521 9d71140 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 |
---
license: apache-2.0
base_model:
- WhiteRabbitNeo/WhiteRabbitNeo-2.5-Qwen-2.5-Coder-7B
- Qwen/Qwen2.5-Coder-7B-Instruct
language:
- en
pipeline_tag: text-generation
library_name: transformers
tags:
- code
- qwen-coder
- finetune
- llama-cpp
- gguf-my-repo
datasets:
- IntelligentEstate/The_Key
---
# TEST
# IntelligentEstate/ReasoningRabbit_Q2.5-Cd-7B-IQ4_XS-GGUF
This model is primarily for use with GPT4ALL and has reasoning capabilities similar to QwQ/o1/03 it was converted to GGUF format using "THE_KEY" dataset for importace matrix Qantization from [`WhiteRabbitNeo/WhiteRabbitNeo-2.5-Qwen-2.5-Coder-7B`](https://huggingface.co/WhiteRabbitNeo/WhiteRabbitNeo-2.5-Qwen-2.5-Coder-7B) using llama.cpp via the ggml.ai's [GGUF-my-repo](https://huggingface.co/spaces/ggml-org/gguf-my-repo) space.
Refer to the [original model card](https://huggingface.co/WhiteRabbitNeo/WhiteRabbitNeo-2.5-Qwen-2.5-Coder-7B) for more details on the model.
# Using with GPT4ALL
after installing GPT4ALL from Nomic download their Reasoner v1 model and familiarize yourself with it, find the storage location under settings and place(only) this GGUF file in the same file as reasoner v1 apply the Jinja template in the "Jinja reasoner" file as well as the chat message above the template adjusting as needed. Enjoy
## Use with llama.cpp
Install llama.cpp through brew (works on Mac and Linux)
```bash
brew install llama.cpp
```
Invoke the llama.cpp server or the CLI.
### CLI:
```bash
llama-cli --hf-repo fuzzy-mittenz/WhiteRabbitNeo-2.5-Qwen-2.5-Coder-7B-IQ4_XS-GGUF --hf-file whiterabbitneo-2.5-qwen-2.5-coder-7b-iq4_xs-imat.gguf -p "The meaning to life and the universe is"
```
### Server:
```bash
llama-server --hf-repo fuzzy-mittenz/WhiteRabbitNeo-2.5-Qwen-2.5-Coder-7B-IQ4_XS-GGUF --hf-file whiterabbitneo-2.5-qwen-2.5-coder-7b-iq4_xs-imat.gguf -c 2048
```
Note: You can also use this checkpoint directly through the [usage steps](https://github.com/ggerganov/llama.cpp?tab=readme-ov-file#usage) listed in the Llama.cpp repo as well.
Step 1: Clone llama.cpp from GitHub.
```
git clone https://github.com/ggerganov/llama.cpp
```
Step 2: Move into the llama.cpp folder and build it with `LLAMA_CURL=1` flag along with other hardware-specific flags (for ex: LLAMA_CUDA=1 for Nvidia GPUs on Linux).
```
cd llama.cpp && LLAMA_CURL=1 make
```
Step 3: Run inference through the main binary.
```
./llama-cli --hf-repo fuzzy-mittenz/WhiteRabbitNeo-2.5-Qwen-2.5-Coder-7B-IQ4_XS-GGUF --hf-file whiterabbitneo-2.5-qwen-2.5-coder-7b-iq4_xs-imat.gguf -p "The meaning to life and the universe is"
```
or
```
./llama-server --hf-repo fuzzy-mittenz/WhiteRabbitNeo-2.5-Qwen-2.5-Coder-7B-IQ4_XS-GGUF --hf-file whiterabbitneo-2.5-qwen-2.5-coder-7b-iq4_xs-imat.gguf -c 2048
``` |