|
--- |
|
base_model: |
|
- beomi/Llama-3-Open-Ko-8B |
|
- ProbeMedicalYonseiMAILab/medllama3-v20 |
|
library_name: transformers |
|
tags: |
|
- mergekit |
|
- merge |
|
license: apache-2.0 |
|
datasets: |
|
- sean0042/KorMedMCQA |
|
--- |
|
# BioLlama-Ko-8B |
|
|
|
|
|
 |
|
|
|
This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit). |
|
|
|
## ๐ Evaluation |
|
|
|
### [kormedmcqa(ํ๊ตญ์ด ์ํ ๋ฒค์น๋งํฌ)](https://huggingface.co/datasets/sean0042/KorMedMCQA) |
|
|
|
| Model | Doctor | Nurse | Pharm | Avg | |
|
|------------------------------------------|-------|-------|-------|-------| |
|
| gpt-4-0613 | 75.09 | 85.86 | 83.22 | 81.39 | |
|
| **iRASC/BioLlama-Ko-8B** | **45.26** | **63.37** | **58.47** | **55.70** | |
|
| gpt-3.5-turbo-0613 | 41.75 | 62.18 | 56.35 | 53.43 | |
|
| llama2-70b | 42.46 | 63.54 | 53.26 | 53.09 | |
|
| SOLAR-10.7B-v1.0 | 37.19 | 55.54 | 54.07 | 48.93 | |
|
| ProbeMedicalYonseiMAILab/medllama3-v20 | 37.19 | 54.68 | 50.65 | 47.51 | |
|
| beomi/Llama-3-Open-Ko-8B | 38.95 | 53.49 | 46.09 | 46.18 | |
|
|
|
|
|
## Merge Details |
|
### Merge Method |
|
|
|
This model was merged using the [DARE](https://arxiv.org/abs/2311.03099) [TIES](https://arxiv.org/abs/2306.01708) merge method using [ProbeMedicalYonseiMAILab/medllama3-v20](https://huggingface.co/ProbeMedicalYonseiMAILab/medllama3-v20) as a base. |
|
|
|
### Models Merged |
|
|
|
The following models were included in the merge: |
|
* [beomi/Llama-3-Open-Ko-8B](https://huggingface.co/beomi/Llama-3-Open-Ko-8B) |
|
|
|
### Configuration |
|
|
|
The following YAML configuration was used to produce this model: |
|
|
|
```yaml |
|
models: |
|
- model: ProbeMedicalYonseiMAILab/medllama3-v20 |
|
- model: beomi/Llama-3-Open-Ko-8B |
|
parameters: |
|
density: 0.8 |
|
weight: 0.45 |
|
merge_method: dare_ties |
|
base_model: ProbeMedicalYonseiMAILab/medllama3-v20 |
|
parameters: |
|
int8_mask: true |
|
dtype: bfloat16 |
|
``` |
