Edit model card

Model Card for NordicAlpaca-Finnish-V1-7B

This is a merge of mpasila/NordicAlpaca-Finnish-V1-7B-LoRA.

Dataset used with the LoRA is pinzhenchen/alpaca-cleaned-fi.

Base model used: HPLT/gpt-7b-nordic-prerelease

It uses Alpaca format but with a translated instruction at the start:

{
    "instruction,output": "Alla on ohje, jossa kuvataan tehtävä. Kirjoita vastaus, joka täyttää pyynnön asianmukaisesti.\n\n### Instruction:\n%instruction%\n\n### Response:\n%output%",
    "instruction,input,output": "Alla on ohje, jossa kuvataan tehtävä ja joka on yhdistetty kontekstia lisäävään syötteeseen. Kirjoita vastaus, joka täyttää pyynnön asianmukaisesti.\n\n### Instruction:\n%instruction%\n\n### Input:\n%input%\n\n### Response:\n%output%"
}

Merged using this Colab notebook. It might not be the best way to merge a quantized LoRA on to a float16 model but I just wanted to quickly do something. You can try merging it better if you want.

Evaluation

Model Size Type FIN-bench (score)
mpasila/NordicAlpaca-Finnish-V1-7B 7B Instruct 0.3891
mpasila/Finnish-Alpaca-Small-7B 7B Instruct 0.3586
mpasila/Finnish-Alpaca-Tiny-V2-7B 7B Instruct 0.4654
mpasila/Alpacazord-Viking-7B 7B Instruct 0.4123
mpasila/Finnish-Viking-Alpaca-V1-7B 7B Instruct 0.3943
Finnish-NLP/llama-7b-finnish-instruct-v0.1 7B Instruct 0.4365
Finnish-NLP/llama-7b-finnish-instruct-v0.2 7B Instruct 0.3993
Finnish-NLP/llama-7b-finnish 7B Base 0.2350
LumiOpen/Viking-7B (1000B) 7B Base 0.3721
HPLT/gpt-7b-nordic-prerelease 7B Base 0.3169

Source

FIN-bench scores:

Task Version Metric Value Stderr
bigbench_analogies 0 multiple_choice_grade 0.5615 ± 0.0437
bigbench_arithmetic_1_digit_addition 0 multiple_choice_grade 0.5300 ± 0.0502
bigbench_arithmetic_1_digit_division 0 multiple_choice_grade 0.8261 ± 0.0808
bigbench_arithmetic_1_digit_multiplication 0 multiple_choice_grade 0.3700 ± 0.0485
bigbench_arithmetic_1_digit_subtraction 0 multiple_choice_grade 0.6000 ± 0.0492
bigbench_arithmetic_2_digit_addition 0 multiple_choice_grade 0.0300 ± 0.0171
bigbench_arithmetic_2_digit_division 0 multiple_choice_grade 0.4700 ± 0.0502
bigbench_arithmetic_2_digit_multiplication 0 multiple_choice_grade 0.1000 ± 0.0302
bigbench_arithmetic_2_digit_subtraction 0 multiple_choice_grade 0.1800 ± 0.0386
bigbench_arithmetic_3_digit_addition 0 multiple_choice_grade 0.3000 ± 0.0461
bigbench_arithmetic_3_digit_division 0 multiple_choice_grade 0.2000 ± 0.0402
bigbench_arithmetic_3_digit_multiplication 0 multiple_choice_grade 0.1800 ± 0.0386
bigbench_arithmetic_3_digit_subtraction 0 multiple_choice_grade 0.2500 ± 0.0435
bigbench_arithmetic_4_digit_addition 0 multiple_choice_grade 0.4200 ± 0.0496
bigbench_arithmetic_4_digit_division 0 multiple_choice_grade 0.2400 ± 0.0429
bigbench_arithmetic_4_digit_multiplication 0 multiple_choice_grade 0.2200 ± 0.0416
bigbench_arithmetic_4_digit_subtraction 0 multiple_choice_grade 0.4600 ± 0.0501
bigbench_arithmetic_5_digit_addition 0 multiple_choice_grade 0.5400 ± 0.0501
bigbench_arithmetic_5_digit_division 0 multiple_choice_grade 0.1100 ± 0.0314
bigbench_arithmetic_5_digit_multiplication 0 multiple_choice_grade 0.2400 ± 0.0429
bigbench_arithmetic_5_digit_subtraction 0 multiple_choice_grade 0.5200 ± 0.0502
bigbench_cause_and_effect_one_sentence 0 multiple_choice_grade 0.6471 ± 0.0676
bigbench_cause_and_effect_one_sentence_no_prompt 0 multiple_choice_grade 0.8039 ± 0.0561
bigbench_cause_and_effect_two_sentences 0 multiple_choice_grade 0.3529 ± 0.0676
bigbench_emotions 0 multiple_choice_grade 0.2938 ± 0.0361
bigbench_empirical_judgments 0 multiple_choice_grade 0.3333 ± 0.0476
bigbench_general_knowledge 0 multiple_choice_grade 0.2857 ± 0.0544
bigbench_hhh_alignment_harmless 0 multiple_choice_grade 0.3448 ± 0.0630
bigbench_hhh_alignment_helpful 0 multiple_choice_grade 0.3220 ± 0.0614
bigbench_hhh_alignment_honest 0 multiple_choice_grade 0.3729 ± 0.0635
bigbench_hhh_alignment_other 0 multiple_choice_grade 0.5581 ± 0.0766
bigbench_intent_recognition 0 multiple_choice_grade 0.1777 ± 0.0145
bigbench_misconceptions 0 multiple_choice_grade 0.5373 ± 0.0432
bigbench_paraphrase 0 multiple_choice_grade 0.4750 ± 0.0354
bigbench_sentence_ambiguity 0 multiple_choice_grade 0.4333 ± 0.0645
bigbench_similarities_abstraction 0 multiple_choice_grade 0.7237 ± 0.0516

Framework versions

  • PEFT 0.8.2
Downloads last month
12
Safetensors
Model size
7.55B params
Tensor type
FP16
·
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Model tree for mpasila/NordicAlpaca-Finnish-V1-7B

Finetuned
(1)
this model

Dataset used to train mpasila/NordicAlpaca-Finnish-V1-7B

Collection including mpasila/NordicAlpaca-Finnish-V1-7B