Update README.md
Browse files
README.md
CHANGED
@@ -63,7 +63,33 @@ The following hyperparameters were used during training:
|
|
63 |
|
64 |
### Training results
|
65 |
|
66 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
67 |
|
68 |
### Framework versions
|
69 |
|
|
|
63 |
|
64 |
### Training results
|
65 |
|
66 |
+
### Performance Metrics
|
67 |
+
AI2SQL's performance was rigorously evaluated post-training. The key metrics used to assess the model were:
|
68 |
+
|
69 |
+
- **Accuracy**: The percentage of queries where the model-generated SQL matched the expected SQL.
|
70 |
+
- **Precision**: The proportion of correctly generated SQL queries out of all queries generated by the model.
|
71 |
+
- **Recall**: The ability of the model to generate all relevant SQL queries corresponding to the input natural language questions.
|
72 |
+
- **F1-Score**: The harmonic mean of precision and recall, providing a balance between the two.
|
73 |
+
|
74 |
+
**Results:**
|
75 |
+
- Accuracy: TBD
|
76 |
+
- Precision: TBD
|
77 |
+
- Recall: TBD
|
78 |
+
- F1-Score: TBD
|
79 |
+
|
80 |
+
### Insights and Observations
|
81 |
+
- **Handling Complex Queries**: AI2SQL demonstrated a high proficiency in handling complex queries involving multiple SQL clauses and parameters.
|
82 |
+
- **Contextual Understanding**: The model showed a notable capability in understanding context and generating SQL queries that accurately reflect nuanced natural language instructions.
|
83 |
+
- **Performance on Diverse Data**: AI2SQL maintained consistent performance across various domains present in the training dataset, indicating its robustness and general applicability.
|
84 |
+
|
85 |
+
### Limitations Observed
|
86 |
+
- **Handling Ambiguous Questions**: The model sometimes struggled with ambiguous natural language inputs where the intent was not clear.
|
87 |
+
- **Query Specificity**: In cases of highly specific queries, the model occasionally generated SQL that was syntactically correct but did not completely align with the nuanced requirements of the question.
|
88 |
+
|
89 |
+
### Future Improvements
|
90 |
+
Based on the training results and observed limitations, future improvements could include:
|
91 |
+
- Enhanced training on ambiguous natural language inputs to improve the model's interpretative capabilities.
|
92 |
+
- Further fine-tuning with a broader range of specific and complex SQL queries to enhance the model's accuracy in generating nuanced SQL statements.
|
93 |
|
94 |
### Framework versions
|
95 |
|