---
license: apache-2.0
---

#  DeciLM-7b-instruct GGUF checkpoints (llama.cpp compatible)


## Chat template
- Feed the chat template to DeciLM-7b-instruct quantized to INT8.
```text
./main -m ~/Downloads/decilm-7b-uniform-gqa-q8_0.gguf -p """
### System:
You are an AI assistant that follows instruction extremely well. Help as much as you can.
### User:
How do I make the most delicious pancakes the world has ever tasted?
### Assistant:
"""
```

- Running on MacBook M2 Pro 32gb::
```
### System:
You are an AI assistant that follows instruction extremely well. Help as much as you can.
### User:
How do I make the most delicious pancakes the world has ever tasted?
### Assistant:
 To make the most delicious pancakes the world ever tasted, follow these steps:

1. In a mixing bowl, combine 2 cups of all-purpose flour, 4 tablespoons of sugar, and 3 teaspoon of baking powder with 1/2 teaspoon salt, mix well.
2. Make a hole in the center and pour in 4 eggs and 1 cup of milk, whisk well mix it until smooth. Add 3 table spoon of oil and a tables of melted butter.
3. Heat your frying pan with little bit butter or oil and ladle batter onto the pan, spread it with 1/2 inch width. Wait for small bubbles to form in the surface and flip over to brown other side until golden.
4. Enjoy your delicious pancakes [end of text]

llama_print_timings:        load time =     343.16 ms
llama_print_timings:      sample time =      14.69 ms /   172 runs   (    0.09 ms per token, 11712.63 tokens per second)
llama_print_timings: prompt eval time =     239.48 ms /    52 tokens (    4.61 ms per token,   217.14 tokens per second)
llama_print_timings:        eval time =    7767.20 ms /   171 runs   (   45.42 ms per token,    22.02 tokens per second)
llama_print_timings:       total time =    8045.89 ms
ggml_metal_free: deallocating
Log end
```