Update README.md
Browse files
README.md
CHANGED
|
@@ -1,3 +1,65 @@
|
|
| 1 |
-
---
|
| 2 |
-
|
| 3 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
base_model: onekq-ai/OneSQL-v0.1-Qwen-7B
|
| 3 |
+
tags:
|
| 4 |
+
- text-generation-inference
|
| 5 |
+
- transformers
|
| 6 |
+
- qwen2
|
| 7 |
+
- gguf
|
| 8 |
+
license: apache-2.0
|
| 9 |
+
language:
|
| 10 |
+
- en
|
| 11 |
+
---
|
| 12 |
+
|
| 13 |
+
# Introduction
|
| 14 |
+
|
| 15 |
+
This model is the GGUF version of [OneSQL-v0.1-Qwen-7B](https://huggingface.co/onekq-ai/OneSQL-v0.1-Qwen-7B). You can also find it on [Ollama](https://ollama.com/onekq/OneSQL-v0.1-Qwen).
|
| 16 |
+
|
| 17 |
+
# Performances
|
| 18 |
+
|
| 19 |
+
The self-evaluation EX score of the original model is **56.01** (compared to **63.33** by the 32B model on the [BIRD leaderboard](https://bird-bench.github.io/).
|
| 20 |
+
Below is the self-evaluation results for each quantization.
|
| 21 |
+
|
| 22 |
+
| Quantization |EX score|
|
| 23 |
+
|------------|------|
|
| 24 |
+
| Q2_K | 29.79 |
|
| 25 |
+
| Q3_K_S | 36.31 |
|
| 26 |
+
| Q3_K_M | 39.24 |
|
| 27 |
+
| Q3_K_L | 40.14 |
|
| 28 |
+
| Q4_1 | 39.06 |
|
| 29 |
+
| Q4_K_S | 42.69 |
|
| 30 |
+
| **Q4_K_M** | **43.95** |
|
| 31 |
+
| Q5_0 | 43.84 |
|
| 32 |
+
| Q5_1 | 41.00 |
|
| 33 |
+
| Q5_K_S | 42.20 |
|
| 34 |
+
| Q5_K_M | 42.07 |
|
| 35 |
+
| Q6_K | 41.68 |
|
| 36 |
+
| Q8_0 | 41.09 |
|
| 37 |
+
|
| 38 |
+
# Quick start
|
| 39 |
+
|
| 40 |
+
To use this model, craft your prompt to start with your database schema in the form of **CREATE TABLE**, followed by your natural language query preceded by **--**.
|
| 41 |
+
Make sure your prompt ends with **SELECT** in order for the model to finish the query for you. There is no need to set other parameters like temperature or max token limit.
|
| 42 |
+
|
| 43 |
+
```sh
|
| 44 |
+
PROMPT="CREATE TABLE students (
|
| 45 |
+
id INTEGER PRIMARY KEY,
|
| 46 |
+
name TEXT,
|
| 47 |
+
age INTEGER,
|
| 48 |
+
grade TEXT
|
| 49 |
+
);
|
| 50 |
+
|
| 51 |
+
-- Find the three youngest students
|
| 52 |
+
SELECT "
|
| 53 |
+
|
| 54 |
+
ollama run onekq-ai/OneSQL-v0.1-Qwen:32B-Q4_K_M "$PROMPT"
|
| 55 |
+
```
|
| 56 |
+
|
| 57 |
+
The model response is the finished SQL query without **SELECT**
|
| 58 |
+
```sql
|
| 59 |
+
* FROM students ORDER BY age ASC LIMIT 3
|
| 60 |
+
```
|
| 61 |
+
|
| 62 |
+
# Caveats
|
| 63 |
+
|
| 64 |
+
* The performance drop from the original model is due to quantization itself, and the lack of beam search support in llama.cpp framework. Use at your own discretion.
|
| 65 |
+
* The Q4_0 quantization suffers from repetitive output token, hence is not recommended for usage.
|