| base_model: amazingvince/openhermes-7b-dpo | |
| license: apache-2.0 | |
| library_name: transformers | |
| tags: | |
| - 4-bit | |
| - AWQ | |
| - text-generation | |
| - autotrain_compatible | |
| - endpoints_compatible | |
| pipeline_tag: text-generation | |
| inference: false | |
| quantized_by: Suparious | |
| # amazingvince/openhermes-7b-dpo AWQ | |
| - Model creator: [amazingvince](https://huggingface.co/amazingvince) | |
| - Original model: [openhermes-7b-dpo](https://huggingface.co/amazingvince/openhermes-7b-dpo) | |
| ## Model Summary | |
| OpenHermes 2.5 Mistral 7B is a state of the art Mistral Fine-tune, a continuation of OpenHermes 2 model, which trained on additional code datasets. | |
| Potentially the most interesting finding from training on a good ratio (est. of around 7-14% of the total dataset) of code instruction was that it has boosted several non-code benchmarks, including TruthfulQA, AGIEval, and GPT4All suite. It did however reduce BigBench benchmark score, but the net gain overall is significant. | |
| Here, we are finetuning openheremes using DPO with various data meant to improve its abilities. | |