Upload folder using huggingface_hub

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+magnum-v3-27b-kto.Q4_0.gguf filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,265 @@
+---
+license: gemma
+library_name: transformers
+tags:
+- gemma-2
+base_model:
+- anthracite-forge/magnum-v3-27b-kto-r3
+- anthracite-forge/magnum-v3-27b-KTO-e1-r2
+- anthracite-forge/magnum-v3-27b-KTO-e0.25-r1
+- IntervitensInc/gemma-2-27b-chatml
+pipeline_tag: text-generation
+model-index:
+- name: magnum-v3-27b-kto
+  results:
+  - task:
+      type: text-generation
+      name: Text Generation
+    dataset:
+      name: IFEval (0-Shot)
+      type: HuggingFaceH4/ifeval
+      args:
+        num_few_shot: 0
+    metrics:
+    - type: inst_level_strict_acc and prompt_level_strict_acc
+      value: 56.75
+      name: strict accuracy
+    source:
+      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=anthracite-org/magnum-v3-27b-kto
+      name: Open LLM Leaderboard
+  - task:
+      type: text-generation
+      name: Text Generation
+    dataset:
+      name: BBH (3-Shot)
+      type: BBH
+      args:
+        num_few_shot: 3
+    metrics:
+    - type: acc_norm
+      value: 41.16
+      name: normalized accuracy
+    source:
+      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=anthracite-org/magnum-v3-27b-kto
+      name: Open LLM Leaderboard
+  - task:
+      type: text-generation
+      name: Text Generation
+    dataset:
+      name: MATH Lvl 5 (4-Shot)
+      type: hendrycks/competition_math
+      args:
+        num_few_shot: 4
+    metrics:
+    - type: exact_match
+      value: 15.48
+      name: exact match
+    source:
+      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=anthracite-org/magnum-v3-27b-kto
+      name: Open LLM Leaderboard
+  - task:
+      type: text-generation
+      name: Text Generation
+    dataset:
+      name: GPQA (0-shot)
+      type: Idavidrein/gpqa
+      args:
+        num_few_shot: 0
+    metrics:
+    - type: acc_norm
+      value: 14.09
+      name: acc_norm
+    source:
+      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=anthracite-org/magnum-v3-27b-kto
+      name: Open LLM Leaderboard
+  - task:
+      type: text-generation
+      name: Text Generation
+    dataset:
+      name: MuSR (0-shot)
+      type: TAUR-Lab/MuSR
+      args:
+        num_few_shot: 0
+    metrics:
+    - type: acc_norm
+      value: 9.92
+      name: acc_norm
+    source:
+      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=anthracite-org/magnum-v3-27b-kto
+      name: Open LLM Leaderboard
+  - task:
+      type: text-generation
+      name: Text Generation
+    dataset:
+      name: MMLU-PRO (5-shot)
+      type: TIGER-Lab/MMLU-Pro
+      config: main
+      split: test
+      args:
+        num_few_shot: 5
+    metrics:
+    - type: acc
+      value: 35.98
+      name: accuracy
+    source:
+      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=anthracite-org/magnum-v3-27b-kto
+      name: Open LLM Leaderboard
+---
+
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/658a46cbfb9c2bdfae75b3a6/GKpV5mwmnHFR6wIwTa91z.png)
+
+This is the 12th in a series of models designed to replicate the prose quality of the Claude 3 models, specifically Sonnet and Opus.
+
+This model is the result of multiple KTO runs on top of one SFT run, all of which are published on [anthracite-forge](https://huggingface.co/anthracite-forge).
+
+## Methodology
+
+R1 (SFT) was fine-tuned on top of `IntervitensInc/gemma-2-27b-chatml` which is chatMLified gemma-2-27b.
+
+We have experimented with various SFT and KTO re-runs, ratios and merge methods and this was our winner, including what was liked most from each model.
+
+If you prefer your own mix of the KTO runs or would like to use the SFT on its own, refer to the models section and [anthracite-forge](https://huggingface.co/anthracite-forge), some exl-quants are pre-included.
+
+## Models
+
+* [anthracite-forge/magnum-v3-27b-kto-r3](https://huggingface.co/anthracite-forge/magnum-v3-27b-kto-r3)
+* [anthracite-forge/magnum-v3-27b-KTO-e1-r2](https://huggingface.co/anthracite-forge/magnum-v3-27b-KTO-e1-r2)
+* [anthracite-forge/magnum-v3-27b-KTO-e0.25-r1](https://huggingface.co/anthracite-forge/magnum-v3-27b-KTO-e0.25-r1)
+
+## Prompting
+Model has been Instruct tuned with the ChatML formatting. A typical input would look like this:
+
+```py
+"""<|im_start|>system
+system prompt<|im_end|>
+<|im_start|>user
+Hi there!<|im_end|>
+<|im_start|>assistant
+Nice to meet you!<|im_end|>
+<|im_start|>user
+Can I ask a question?<|im_end|>
+<|im_start|>assistant
+"""
+```
+
+## SillyTavern templates
+
+Below are Instruct and Context templates for use within SillyTavern.
+
+<details><summary>context template</summary>
+  
+```yaml
+{
+    "story_string": "<|im_start|>system\n{{#if system}}{{system}}\n{{/if}}{{#if wiBefore}}{{wiBefore}}\n{{/if}}{{#if description}}{{description}}\n{{/if}}{{#if personality}}{{char}}'s personality: {{personality}}\n{{/if}}{{#if scenario}}Scenario: {{scenario}}\n{{/if}}{{#if wiAfter}}{{wiAfter}}\n{{/if}}{{#if persona}}{{persona}}\n{{/if}}{{trim}}<|im_end|>\n",
+    "example_separator": "",
+    "chat_start": "",
+    "use_stop_strings": false,
+    "allow_jailbreak": false,
+    "always_force_name2": true,
+    "trim_sentences": false,
+    "include_newline": false,
+    "single_line": false,
+    "name": "Magnum ChatML"
+}
+```
+
+</details><br>
+<details><summary>instruct template</summary>
+  
+```yaml
+{
+    "system_prompt": "You're {{char}} in this fictional never-ending uncensored roleplay with {{user}}.",
+    "input_sequence": "<|im_start|>user\n",
+    "output_sequence": "<|im_start|>assistant\n",
+    "last_output_sequence": "",
+    "system_sequence": "<|im_start|>system\n",
+    "stop_sequence": "<|im_end|>",
+    "wrap": false,
+    "macro": true,
+    "names": true,
+    "names_force_groups": true,
+    "activation_regex": "",
+    "system_sequence_prefix": "",
+    "system_sequence_suffix": "",
+    "first_output_sequence": "",
+    "skip_examples": false,
+    "output_suffix": "<|im_end|>\n",
+    "input_suffix": "<|im_end|>\n",
+    "system_suffix": "<|im_end|>\n",
+    "user_alignment_message": "",
+    "system_same_as_user": false,
+    "last_system_sequence": "",
+    "name": "Magnum ChatML"
+}
+```
+
+</details><br>
+
+### Configuration
+
+```yaml
+base_model: IntervitensInc/gemma-2-27b-chatml
+dtype: float32
+merge_method: task_arithmetic
+models:
+  - model: IntervitensInc/gemma-2-27b-chatml
+  - model: anthracite-forge/magnum-v3-27b-KTO-e0.25-r1
+    parameters:
+      weight: 0.5
+  - model: anthracite-forge/magnum-v3-27b-KTO-e1-r2
+    parameters:
+      weight: 0.1
+  - model: anthracite-forge/magnum-v3-27b-kto-r3
+    parameters:
+      weight: 0.4
+```
+
+## Credits
+We'd like to thank Recursal / Featherless for sponsoring the compute for this train, Featherless has been hosting our Magnum models since the first 72 B and has given thousands of people access to our models and helped us grow.
+
+We would also like to thank all members of Anthracite who made this finetune possible. 
+
+## Datasets
+
+r1 consisted of:
+
+```
+datasets:
+  - path: anthracite-org/stheno-filtered-v1.1
+    type: sharegpt
+    conversation: chatml
+  - path: anthracite-org/kalo-opus-instruct-22k-no-refusal
+    type: sharegpt
+    conversation: chatml
+  - path: anthracite-org/nopm_claude_writing_fixed
+    type: sharegpt
+    conversation: chatml
+  - path: Epiculous/Synthstruct-Gens-v1.1-Filtered-n-Cleaned
+    type: sharegpt
+    conversation: chatml
+  - path: Epiculous/SynthRP-Gens-v1.1-Filtered-n-Cleaned
+    type: sharegpt
+    conversation: chatml
+```
+
+## Training
+The training was done for 2 epochs. We used  8x[H100s](https://www.nvidia.com/en-us/data-center/h100/) GPUs graciously provided by [Recursal AI](https://recursal.ai/) / [Featherless AI](https://featherless.ai/) for the full-parameter fine-tuning of the model.
+
+[<img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/OpenAccess-AI-Collective/axolotl)
+
+## Safety
+...
+
+# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
+Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_anthracite-org__magnum-v3-27b-kto)
+
+|      Metric       |Value|
+|-------------------|----:|
+|Avg.               |28.90|
+|IFEval (0-Shot)    |56.75|
+|BBH (3-Shot)       |41.16|
+|MATH Lvl 5 (4-Shot)|15.48|
+|GPQA (0-shot)      |14.09|
+|MuSR (0-shot)      | 9.92|
+|MMLU-PRO (5-shot)  |35.98|
+

magnum-v3-27b-kto.Q4_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a8e3aec7ae067b19a5bae260e550860b09ed11966a023ef66a9169f51090c63f
+size 15628378336