README.md

---
base_model: onekq-ai/OneSQL-v0.1-Qwen-7B
tags:
- text-generation-inference
- transformers
- qwen2
- gguf
license: apache-2.0
language:
- en
---

# Introduction

This model is the GGUF version of [OneSQL-v0.1-Qwen-7B](https://huggingface.co/onekq-ai/OneSQL-v0.1-Qwen-7B). You can also find it on [Ollama](https://ollama.com/onekq/OneSQL-v0.1-Qwen).

# Performances

The self-evaluation EX score of the original model is **56.19** (compared to **63.33** by the 32B model on the [BIRD leaderboard](https://bird-bench.github.io/).
Below is the self-evaluation results for each quantization.

| Quantization |EX score|
|------------|------|
| Q2_K       | 29.79 |
| Q3_K_S     | 36.31 |
| Q3_K_M     | 39.24 |
| Q3_K_L     | 40.14 |
| Q4_1       | 39.06 |
| Q4_K_S     | 42.69 |
| **Q4_K_M** | **43.95** |
| Q5_0       | 43.84 |
| Q5_1       | 41.00 |
| Q5_K_S     | 42.20 |
| Q5_K_M     | 42.07 |
| Q6_K       | 41.68 |
| Q8_0       | 41.09 |

# Quick start

To use this model, craft your prompt to start with your database schema in the form of **CREATE TABLE**, followed by your natural language query preceded by **--**.
Make sure your prompt ends with **SELECT** in order for the model to finish the query for you. There is no need to set other parameters like temperature or max token limit.

```sh
PROMPT="CREATE TABLE students (
    id INTEGER PRIMARY KEY,
    name TEXT,
    age INTEGER,
    grade TEXT
);

-- Find the three youngest students
SELECT "

ollama run onekq-ai/OneSQL-v0.1-Qwen:32B-Q4_K_M "$PROMPT"
```

The model response is the finished SQL query without **SELECT**
```sql
* FROM students ORDER BY age ASC LIMIT 3
```

# Caveats

* The performance drop from the original model is due to quantization itself, and the lack of beam search support in llama.cpp framework. Use at your own discretion.
* The Q4_0 quantization suffers from repetitive output token, hence is not recommended for usage.