README.md

---
base_model: unsloth/qwen2.5-coder-7b-bnb-4bit
library_name: peft
license: apache-2.0
datasets:
- WPAI-INC/wp-sql-instruction-pairs
tags:
- wordpress
- sql
- wpaigpt
- text2sql
---

WPAIGPT-SQL-01 is a specialized text-to-SQL model designed for WordPress and WordPress plugins. It generates SQL queries based on natural language requests, with a focus on WordPress-specific database structures and popular plugins.

## Model Details

### Model Description

WPAIGPT-SQL-01 is a fine-tuned version of the Qwen2.5-Coder-7B model, optimized for generating SQL queries for WordPress databases. It can handle queries related to core WordPress tables as well as tables added by various plugins.

- **Developed by:** [WPAI Inc](https://wpai.co), James LePage
- **Funded by:** WPAI Inc
- **Model type:** Text-to-SQL Language Model
- **Language(s) (NLP):** English
- **License:** Apache 2.0
- **Finetuned from model:** Qwen2.5-Coder-7B-Instruct

## Uses

### Direct Use

The model is designed for direct text-to-SQL generation for WordPress databases. Users can input natural language requests, optionally including plugin names, versions, and table descriptions, to generate SQL queries. This is particularly useful for:

1. Retrieving information from WordPress databases
2. Adding functionality to existing WordPress plugins by generating SQL queries
3. Assisting developers in creating database queries for WordPress projects

### Downstream Use

1. Integration into WPAI products, primarily [AgentWP](https://agentwp.com), for real-time information retrieval from WordPress websites
2. Use in code generation tools to create queries for more complete WordPress systems like plugins
3. Incorporation into agent pipelines for WordPress-related tasks

### Out-of-Scope Use

While there are no strict out-of-scope uses, users should be aware that as a Transformer-based model, it can potentially hallucinate or generate incorrect queries. All generated SQL should be verified before execution against a live WordPress database.

## Bias, Risks, and Limitations

- The model may be biased towards more popular WordPress plugins and those with more extensive database interactions.
- There's a bias towards SELECT and read-only operations over database-modifying queries.
- The model's knowledge is limited to the training data, which may not cover all possible WordPress plugins or database structures.
- As with any language model, there's a risk of generating syntactically correct but logically incorrect or potentially harmful SQL queries.

### Recommendations

- Always verify and test generated SQL queries before executing them on a live WordPress database.
- Use in conjunction with proper access controls and user authentication to prevent unauthorized database access.
- Regularly update the model to include knowledge of new WordPress versions and popular plugins.
- Implement additional safety checks and validations when using the model in automated systems.

## Training Details

### Training Data

The training data consists of hundreds of thousands of instruction-to-SQL examples, structured as follows:
- 25% include described tables that WordPress plugins may add, along with plugin name, version, and instruction
- 25% include only the plugin name, version, and instruction
- 50% include only the instruction

The queries are derived from popular WordPress plugins, both from the official WordPress repository and premium plugins. The data generation process involves:
1. Indexing plugin codebases
2. Extracting code that manipulates the WordPress database
3. Synthetically generating SQL queries
4. Verifying queries by running them against a WordPress installation with the plugin installed

There's a bias towards the most popular WordPress plugins and those with significant database interactions. Additional manual data has been included for specific plugins like WooCommerce, LearnDash, and Gravity Forms.

### Training Procedure

The training procedure details are available in the provided Python notebook. For specific information about hyperparameters, preprocessing steps, and other training details, please refer to the notebook.

## Evaluation

### Testing Data, Factors & Metrics

Formal evaluations have not been conducted. The model's performance is primarily assessed through:
1. A/B testing in WPAI products
2. User rankings on end systems (AgentWP, [CodeWP](https://codewp.ai), and other WPAI products)

## Technical Specifications

### Model Architecture and Objective

The model is based on the Qwen2.5-Coder-7B architecture, fine-tuned for the specific task of WordPress SQL generation. It uses a causal language modeling objective to generate SQL queries based on natural language inputs.

Key features of the base Qwen2.5-Coder-7B model include:
- Number of Parameters: 7.61B
- Number of Layers: 28
- Number of Attention Heads: 28 for Q and 4 for KV (using Grouped-Query Attention)
- Context Length: Full 131,072 tokens (with the ability to handle long contexts using YaRN technique)

The model has been specifically fine-tuned to understand WordPress database structures and generate appropriate SQL queries, maintaining its coding capabilities while focusing on the WordPress ecosystem.