BEE-spoke-data/smol_llama-220M-open_instruct
Please note that this is an experiment, and the model has limitations because it is smol.
prompt format is alpaca.
Below is an instruction that describes a task, paired with an input that
provides further context. Write a response that appropriately completes
the request.
### Instruction:
How can I increase my meme production/output? Currently, I only create them in ancient babylonian which is time consuming.
### Response:
This was not trained using a separate 'inputs' field (as VMware/open-instruct
doesn't use one).
Example
Output on the text above ^. The inference API is set to sample with low temp so you should see (at least slightly) different generations each time.
Note that the inference API parameters used here are an initial educated guess, and may be updated over time:
inference:
parameters:
do_sample: true
renormalize_logits: true
temperature: 0.25
top_p: 0.95
top_k: 50
min_new_tokens: 2
max_new_tokens: 96
repetition_penalty: 1.04
no_repeat_ngram_size: 6
epsilon_cutoff: 0.0006
Feel free to experiment with the parameters using the model in Python and let us know if you have improved results with other params!
Data
This was trained on VMware/open-instruct
so do whatever you want, provided it falls under the base apache-2.0 license :)
Open LLM Leaderboard Evaluation Results
Detailed results can be found here
Metric | Value |
---|---|
Avg. | 29.19 |
AI2 Reasoning Challenge (25-Shot) | 25.00 |
HellaSwag (10-Shot) | 29.71 |
MMLU (5-Shot) | 26.11 |
TruthfulQA (0-shot) | 44.06 |
Winogrande (5-shot) | 50.28 |
GSM8k (5-shot) | 0.00 |
- Downloads last month
- 32
Model tree for BEE-spoke-data/smol_llama-220M-open_instruct
Dataset used to train BEE-spoke-data/smol_llama-220M-open_instruct
Collection including BEE-spoke-data/smol_llama-220M-open_instruct
Evaluation results
- normalized accuracy on AI2 Reasoning Challenge (25-Shot)test set Open LLM Leaderboard25.000
- normalized accuracy on HellaSwag (10-Shot)validation set Open LLM Leaderboard29.710
- accuracy on MMLU (5-Shot)test set Open LLM Leaderboard26.110
- mc2 on TruthfulQA (0-shot)validation set Open LLM Leaderboard44.060
- accuracy on Winogrande (5-shot)validation set Open LLM Leaderboard50.280
- accuracy on GSM8k (5-shot)test set Open LLM Leaderboard0.000