| license: llama3.2 | |
| datasets: | |
| - teknium/OpenHermes-2.5 | |
| - NousResearch/hermes-function-calling-v1 | |
| base_model: | |
| - minpeter/QLoRA-Llama-3.2-1B-chatml-tool-v2 | |
| - meta-llama/Llama-3.2-1B | |
| language: | |
| - en | |
| pipeline_tag: text-generation | |
| library_name: transformers | |
| tags: | |
| - axolotl | |
| - merge | |
| ## Model Performance Comparison (BFCL) | |
| | task name | minpeter/Llama-3.2-1B-chatml-tool-v4 | meta-llama/Llama-3.2-1B-Instruct (measure) | meta-llama/Llama-3.2-1B-Instruct (Reported) | | |
| |-----------------|-----------------------------------|-----------------------------------|---------------------------------------| | |
| | parallel_multiple | 0.000 | 0.025 | **0.15** | | |
| | parallel | 0.000 | 0.035 | **0.36** | | |
| | simple | **0.7725** | 0.215 | 0.2925 | | |
| | multiple | **0.765** | 0.17 | 0.335 | | |
| *Parallel calls are not taken into account. 0 points are expected. We plan to fix this in next version.* |