nielsr HF Staff commited on
Commit
67f8371
Β·
verified Β·
1 Parent(s): 337bcb3

Add comprehensive model card for CSC-SQL model

Browse files

This PR adds a comprehensive model card for the `CscSQL-Grpo-Qwen2.5-Coder-7B-Instruct` model, based on the paper [CSC-SQL: Corrective Self-Consistency in Text-to-SQL via Reinforcement Learning](https://huggingface.co/papers/2505.13271).

The model card includes:
- Essential metadata such as `pipeline_tag: text-generation` and `library_name: transformers` to ensure the model is easily discoverable and correctly categorized on the Hugging Face Hub.
- Relevant tags like `text-to-sql`, `sql`, and `qwen2`, along with `datasets` and `base_model` for better context.
- A link to the official paper and its abstract.
- A link to the GitHub repository for access to the code and more project details.
- An overview of the model's approach, main results, and a table of related models and datasets.
- A clear "How to Use" section with a Python code snippet demonstrating inference with the `transformers` library, tailored for Text-to-SQL generation using the model's specific chat template.
- The BibTeX citation for proper academic attribution.

This update significantly improves the model's documentation, making it more accessible and user-friendly on the Hugging Face Hub.

Files changed (1) hide show
  1. README.md +141 -0
README.md ADDED
@@ -0,0 +1,141 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: cc-by-nc-4.0
3
+ pipeline_tag: text-generation
4
+ library_name: transformers
5
+ tags:
6
+ - text-to-sql
7
+ - sql
8
+ - qwen2
9
+ datasets:
10
+ - cycloneboy/bird_train
11
+ base_model: Qwen/Qwen2.5-7B-Instruct
12
+ ---
13
+
14
+ # CSC-SQL: Corrective Self-Consistency in Text-to-SQL via Reinforcement Learning
15
+
16
+ This repository contains the `CscSQL-Grpo-Qwen2.5-Coder-7B-Instruct` model, presented in the paper [CSC-SQL: Corrective Self-Consistency in Text-to-SQL via Reinforcement Learning](https://huggingface.co/papers/2505.13271).
17
+
18
+ ## Abstract
19
+
20
+ Large language models (LLMs) have demonstrated strong capabilities in translating natural language questions about relational databases into SQL queries. In particular, test-time scaling techniques such as Self-Consistency and Self-Correction can enhance SQL generation accuracy by increasing computational effort during inference. However, these methods have notable limitations: Self-Consistency may select suboptimal outputs despite majority votes, while Self-Correction typically addresses only syntactic errors. To leverage the strengths of both approaches, we propose CSC-SQL, a novel method that integrates Self-Consistency and Self-Correction. CSC-SQL selects the two most frequently occurring outputs from parallel sampling and feeds them into a merge revision model for correction. Additionally, we employ the Group Relative Policy Optimization (GRPO) algorithm to fine-tune both the SQL generation and revision models via reinforcement learning, significantly enhancing output quality. Experimental results confirm the effectiveness and generalizability of CSC-SQL. On the BIRD private test set, our 7B model achieves 71.72% execution accuracy, while the 32B model achieves 73.67%.
21
+
22
+ ## Code
23
+
24
+ The official implementation, including training and evaluation scripts, can be found on GitHub: [https://github.com/CycloneBoy/csc_sql](https://github.com/CycloneBoy/csc_sql)
25
+
26
+ ## Introduction
27
+
28
+ CSC-SQL is a novel method that integrates Self-Consistency and Self-Correction to enhance SQL generation accuracy. It addresses the limitations of existing test-time scaling techniques by combining their strengths. The method involves selecting the two most frequently occurring outputs from parallel sampling and feeding them into a merge revision model for correction. Furthermore, the Group Relative Policy Optimization (GRPO) algorithm is employed to fine-tune both the SQL generation and revision models via reinforcement learning, leading to significantly enhanced output quality.
29
+
30
+ The framework overview is illustrated below:
31
+
32
+ ![csc_sql_framework](https://github.com/CycloneBoy/csc_sql/raw/main/data/image/csc_sql_framework.png)
33
+
34
+ ## Main Results
35
+
36
+ The CSC-SQL model achieves state-of-the-art results in Text-to-SQL generation. On the BIRD private test set, the 7B model achieves 71.72% execution accuracy, while the 32B model achieves 73.67%.
37
+
38
+ Performance Comparison of different Text-to-SQL methods on BIRD dev and test dataset:
39
+ <img src="https://github.com/CycloneBoy/csc_sql/raw/main/data/image/csc_sql_result_main.png" height="500" alt="Performance Comparison">
40
+
41
+ ## Models and Datasets
42
+
43
+ The project provides various models and datasets, which can be found on Hugging Face and ModelScope:
44
+
45
+ | **Model and Dataset** | Modelscope | HuggingFace |
46
+ |---------------------------------------|-------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------|
47
+ | bird train and dev dataset | [πŸ€– Modelscope](https://modelscope.cn/datasets/cycloneboy/bird_train) | [πŸ€— HuggingFace](https://huggingface.co/datasets/cycloneboy/bird_train) |
48
+ | CscSQL-Merge-Qwen2.5-Coder-3B-Instruct | [πŸ€– Modelscope](https://modelscope.cn/models/cycloneboy/CscSQL-Merge-Qwen2.5-Coder-3B-Instruct) | [πŸ€— HuggingFace](https://huggingface.co/cycloneboy/CscSQL-Merge-Qwen2.5-Coder-3B-Instruct) |
49
+ | CscSQL-Merge-Qwen2.5-Coder-7B-Instruct | [πŸ€– Modelscope](https://modelscope.cn/models/cycloneboy/CscSQL-Merge-Qwen2.5-Coder-7B-Instruct) | [πŸ€— HuggingFace](https://huggingface.co/cycloneboy/CscSQL-Merge-Qwen2.5-Coder-7B-Instruct) |
50
+ | CscSQL-Grpo-Qwen2.5-Coder-3B-Instruct | [πŸ€– Modelscope](https://modelscope.cn/models/cycloneboy/CscSQL-Grpo-Qwen2.5-Coder-3B-Instruct) | [πŸ€— HuggingFace](https://huggingface.co/cycloneboy/CscSQL-Grpo-Qwen2.5-Coder-3B-Instruct) |
51
+ | CscSQL-Grpo-XiYanSQL-QwenCoder-3B-2502 | [πŸ€– Modelscope](https://modelscope.cn/models/cycloneboy/CscSQL-Grpo-XiYanSQL-QwenCoder-3B-2502) | [πŸ€— HuggingFace](https://huggingface.co/cycloneboy/CscSQL-Grpo-XiYanSQL-QwenCoder-3B-2502) |
52
+ | CscSQL-Grpo-Qwen2.5-Coder-7B-Instruct | [πŸ€– Modelscope](https://modelscope.cn/models/cycloneboy/CscSQL-Grpo-Qwen2.5-Coder-7B-Instruct) | [πŸ€— HuggingFace](https://huggingface.co/cycloneboy/CscSQL-Grpo-Qwen2.5-Coder-7B-Instruct) |
53
+ | CscSQL-Grpo-XiYanSQL-QwenCoder-7B-2502 | [πŸ€– Modelscope](https://modelscope.cn/models/cycloneboy/CscSQL-Grpo-XiYanSQL-QwenCoder-7B-2502) | [πŸ€— HuggingFace](https://huggingface.co/cycloneboy/CscSQL-Grpo-XiYanSQL-QwenCoder-7B-2502) |
54
+
55
+ ## Usage
56
+
57
+ You can use this model with the Hugging Face `transformers` library. Here's a quick example for Text-to-SQL generation following the Qwen chat template:
58
+
59
+ ```python
60
+ import torch
61
+ from transformers import AutoTokenizer, AutoModelForCausalLM, GenerationConfig
62
+
63
+ model_name = "cycloneboy/CscSQL-Grpo-Qwen2.5-Coder-7B-Instruct"
64
+
65
+ tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
66
+ model = AutoModelForCausalLM.from_pretrained(
67
+ model_name,
68
+ device_map="auto",
69
+ torch_dtype=torch.bfloat16,
70
+ trust_remote_code=True
71
+ ).eval()
72
+
73
+ # Example natural language question and a simplified database schema
74
+ question = "List the names of all employees who work in the 'Sales' department."
75
+ schema = """
76
+ CREATE TABLE employees (
77
+ employee_id INT PRIMARY KEY,
78
+ name VARCHAR(255),
79
+ department_id INT
80
+ );
81
+
82
+ CREATE TABLE departments (
83
+ department_id INT PRIMARY KEY,
84
+ department_name VARCHAR(255)
85
+ );
86
+ """
87
+
88
+ # Construct the prompt according to the model's expected input format for Text-to-SQL
89
+ # This is typically a combination of natural language question and the schema
90
+ user_prompt = f"Question: {question}
91
+ Schema: {schema}
92
+ SQL:"
93
+
94
+ messages = [
95
+ {"role": "user", "content": user_prompt}
96
+ ]
97
+
98
+ # Apply the chat template to format the input for the model
99
+ text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
100
+ model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
101
+
102
+ # Define generation configuration
103
+ generation_config = GenerationConfig(
104
+ do_sample=True,
105
+ temperature=0.7,
106
+ top_p=0.8,
107
+ top_k=20,
108
+ repetition_penalty=1.05,
109
+ max_new_tokens=512, # Adjust as needed for SQL query length
110
+ eos_token_id=tokenizer.eos_token_id,
111
+ pad_token_id=tokenizer.pad_token_id,
112
+ )
113
+
114
+ # Generate the SQL query
115
+ generated_ids = model.generate(
116
+ model_inputs.input_ids,
117
+ generation_config=generation_config
118
+ )
119
+
120
+ # Decode the generated SQL, skipping the input prompt
121
+ generated_sql = tokenizer.batch_decode(generated_ids[:, model_inputs.input_ids.shape[1]:], skip_special_tokens=True)[0]
122
+
123
+ print("Generated SQL Query:")
124
+ print(generated_sql)
125
+ ```
126
+
127
+ ## Citation
128
+
129
+ If you find our work helpful or inspiring, please feel free to cite it:
130
+
131
+ ```bibtex
132
+ @misc{sheng2025cscsqlcorrectiveselfconsistencytexttosql,
133
+ title={CSC-SQL: Corrective Self-Consistency in Text-to-SQL via Reinforcement Learning},
134
+ author={Lei Sheng and Shuai-Shuai Xu},
135
+ year={2025},
136
+ eprint={2505.13271},
137
+ archivePrefix={arXiv},
138
+ primaryClass={cs.CL},
139
+ url={https://arxiv.org/abs/2505.13271},
140
+ }
141
+ ```