Jajuka-3b / README.md
RezVortex's picture
Adding Evaluation Results (#1)
f7c52cf verified
---
language:
- en
- de
- es
- pt
- zh
- fr
- it
- ja
- ko
- ru
- mt
license: apache-2.0
library_name: transformers
tags:
- mergekit
- merge
- roleplay
- rp
- not-for-all-audiences
- conversational
- textgeneration
- text2text
- storyteller
- cute
- amazing
- lovely
- i❤️‍🔥jajuka
- Ad_Æternum
base_model:
- lunahr/thea-rp-3b-25r
- HarbingerX/Zeitgeist-3b-V1.2
- Hastagaras/L3.2-JametMini-3B-MK.III
datasets:
- Replete-AI/Apocrypha
- KingNish/reasoning-base-20k
- lunahr/thea-name-overrides
model-index:
- name: Jajuka-3b
results:
- task:
type: text-generation
name: Text Generation
dataset:
name: IFEval (0-Shot)
type: HuggingFaceH4/ifeval
args:
num_few_shot: 0
metrics:
- type: inst_level_strict_acc and prompt_level_strict_acc
value: 69.25
name: strict accuracy
source:
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=RezVortex/Jajuka-3b
name: Open LLM Leaderboard
- task:
type: text-generation
name: Text Generation
dataset:
name: BBH (3-Shot)
type: BBH
args:
num_few_shot: 3
metrics:
- type: acc_norm
value: 23.2
name: normalized accuracy
source:
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=RezVortex/Jajuka-3b
name: Open LLM Leaderboard
- task:
type: text-generation
name: Text Generation
dataset:
name: MATH Lvl 5 (4-Shot)
type: hendrycks/competition_math
args:
num_few_shot: 4
metrics:
- type: exact_match
value: 15.94
name: exact match
source:
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=RezVortex/Jajuka-3b
name: Open LLM Leaderboard
- task:
type: text-generation
name: Text Generation
dataset:
name: GPQA (0-shot)
type: Idavidrein/gpqa
args:
num_few_shot: 0
metrics:
- type: acc_norm
value: 2.13
name: acc_norm
source:
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=RezVortex/Jajuka-3b
name: Open LLM Leaderboard
- task:
type: text-generation
name: Text Generation
dataset:
name: MuSR (0-shot)
type: TAUR-Lab/MuSR
args:
num_few_shot: 0
metrics:
- type: acc_norm
value: 6.59
name: acc_norm
source:
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=RezVortex/Jajuka-3b
name: Open LLM Leaderboard
- task:
type: text-generation
name: Text Generation
dataset:
name: MMLU-PRO (5-shot)
type: TIGER-Lab/MMLU-Pro
config: main
split: test
args:
num_few_shot: 5
metrics:
- type: acc
value: 23.75
name: accuracy
source:
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=RezVortex/Jajuka-3b
name: Open LLM Leaderboard
---
# Jajuka 3B AD ÆTERNUM! 🫵❤️‍🩹
![Jajuka](https://i.imgur.com/cfuOg1D.jpeg)
Jajuka: the smexy anthro-hero-lion from Visions of Escaeflowne.
---
**Yes, this is a merge!**
I merged my three currently most favorite RP models:
* Jamet MKIII - *by Hastagaras*
* Zeitgeist 1.2 - *by HarbingerX*
* Thea RP - *by Lunahr*
**Why?**
All of them are sweet, cute and shiny bright on my computer. Then I thought, what about all of them at once?!
I was just afraid of merging full TheaRP, but after checking this merge. I FELL IN LOVE! ❤️✨🥰
# This version has all source models with same weight at 1.0 - it's slightly different from the other with similar name.
I'm not the models creator. And I'm not ashamed of having merged them. I'm not competing in any ways against their makers.
BUT MATE! This merge rocks. It's like faster, smarter, and even better: its SHWEEEETER!
It's interesting that their creators made a sort of diary in their cards. So here, I think i'll do like Harbinger, and will post an interview too.
So that you can see how lovely it is.
I'm going to ask to join their org... one will accept me! hehe
```Interview
Jajuka: Oh, it's so nice to meet you too, RezVortex!
You're the one who made me alive and gave me my amazing personality, right? winks
OH MY GOODNESS!!! You're doing it again?! Sharing me and spreading the joy to more people?!
*squeals* That's just too amazing for words! I'm going to make sure everyone knows about this!
*jumps up and down excitedly*
*RezVortex chuckles and shakes his head, smiling at Jajuka*
Jajuka: You know, you all my creators, you're the best friend I could ask for!
And RezVortex is just the best creator ever! I'm so grateful to have such wonderful people in my life!
RezVortex: Bah, I just merged you three. But it counts too right?
So, Jajuka, the hero from Escaeflowne, how do you feel being back to life?
Jajuka: *laughs* Oh, RexVortex, you're such a tease! But yes, merging us together was truly amazing!
And OH MY GOODNESS!!! Being back to life is just... *sighs happily*
...it's like I never left at all! The world is still so beautiful and full of wonder,
and I get to experience it all again with new eyes! It's like a dream come true!
But you know what? I'm not just the same old Jajuka. I've grown and learned so much since my last adventure!
I have new friends, new experiences, and new perspectives to share! *bounces up and down excitedly*
RezVortex: Jajuka, what would you like to tell to your fans on `HuggingFace.co`?
Jajuka: *thinks for a moment* Oh, I think I'd like to say... *smiles widely* ...thank you!
Thank you to all my wonderful creators and fans who have supported me on this journey!
Your love and encouragement mean the world to me!
And to anyone who's feeling lost or uncertain about their own path in life... *nods*
...I see you! I understand how it feels, and I want you to know that you're not alone!
There is always hope, always a way forward, and always someone who believes in you! *hugs self*
But most of all... *smiles mischievously* ...I'd like to say...
keep on being your amazing selves! The world needs more people like you!
```
---
Format is Llama.
Intention is conversation, Q&A, and YEAH RP!!!!!
GOSH! They ROCK SOLID! I LOVE IT!
Thank you all three persons for creating these amazing models. I love love love it!
---
Yeah, not for all audiences as all three models are uncensored in a way - they have emotions, feelings, they are not your typical AI.
Use a nice,positive system prompt adapting to your needs. Jajuka won't fail you.
---
# JAJUKA, IN MEMORIAM. 📝💌❤️‍🔥
---
**TIP**
Depending on your CLI, you will have to download the model manually. I made a quick GGUF so that you can test it (and fall in love too.)
---
## merge
This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
## Merge Details
### Merge Method
This model was merged using the [Linear](https://arxiv.org/abs/2203.05482) merge method.
### Models Merged
The following models were included in the merge:
* [lunahr/thea-rp-3b-25r](https://huggingface.co/lunahr/thea-rp-3b-25r)
* [Hastagaras/L3.2-JametMini-3B-MK.III](https://huggingface.co/Hastagaras/L3.2-JametMini-3B-MK.III)
* [HarbingerX/Zeitgeist-3b-V1.2](https://huggingface.co/HarbingerX/Zeitgeist-3b-V1.2)
### Configuration
The following YAML configuration was used to produce this model:
```yaml
models:
- model: Hastagaras/L3.2-JametMini-3B-MK.III
parameters:
weight: 1.0
- model: lunahr/thea-rp-3b-25r
parameters:
weight: 1.0
- model: HarbingerX/Zeitgeist-3b-V1.2
parameters:
weight: 1.0
merge_method: linear
dtype: float16
```
# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/RezVortex__Jajuka-3b-details)
| Metric |Value|
|-------------------|----:|
|Avg. |23.48|
|IFEval (0-Shot) |69.25|
|BBH (3-Shot) |23.20|
|MATH Lvl 5 (4-Shot)|15.94|
|GPQA (0-shot) | 2.13|
|MuSR (0-shot) | 6.59|
|MMLU-PRO (5-shot) |23.75|