Update README.md
Browse files
README.md
CHANGED
@@ -2,40 +2,51 @@
|
|
2 |
license: apache-2.0
|
3 |
---
|
4 |
|
5 |
-
#
|
|
|
|
|
6 |
|
7 |
-
|
8 |
-
## Chat template
|
9 |
-
- Feed the chat template to DeciLM-7b-instruct quantized to INT8.
|
10 |
-
```text
|
11 |
-
./main -m ~/Downloads/decilm-7b-uniform-gqa-q8_0.gguf -p """
|
12 |
-
### System:
|
13 |
-
You are an AI assistant that follows instruction extremely well. Help as much as you can.
|
14 |
-
### User:
|
15 |
-
How do I make the most delicious pancakes the world has ever tasted?
|
16 |
-
### Assistant:
|
17 |
-
"""
|
18 |
```
|
19 |
-
|
20 |
-
- Running on MacBook M2 Pro 32gb::
|
21 |
```
|
22 |
-
### System:
|
23 |
-
You are an AI assistant that follows instruction extremely well. Help as much as you can.
|
24 |
-
### User:
|
25 |
-
How do I make the most delicious pancakes the world has ever tasted?
|
26 |
-
### Assistant:
|
27 |
-
To make the most delicious pancakes the world ever tasted, follow these steps:
|
28 |
|
29 |
-
|
30 |
-
|
31 |
-
|
32 |
-
|
33 |
|
34 |
-
|
35 |
-
|
36 |
-
|
37 |
-
|
38 |
-
|
39 |
-
|
40 |
-
|
41 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2 |
license: apache-2.0
|
3 |
---
|
4 |
|
5 |
+
# DeciLM-7b-instruct GGUF checkpoints
|
6 |
+
This repository includes DeciLM-7b-instruct checkpoints in the GGUF format.<br>
|
7 |
+
DeciLM performs well on commodity CPUs using the llama.cpp codebase.
|
8 |
|
9 |
+
## 1. Clone and build llama.cpp (1 minute)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
10 |
```
|
11 |
+
git clone https://github.com/ggerganov/llama.cpp && cd llama.cpp && make -j
|
|
|
12 |
```
|
|
|
|
|
|
|
|
|
|
|
|
|
13 |
|
14 |
+
## 2. Download the GGUF checkpoint
|
15 |
+
- Go to "Files"
|
16 |
+
- Click on "decilm-7b-uniform-gqa-q8_0.gguf"
|
17 |
+
- Click on the "Download" button
|
18 |
|
19 |
+
## 3. Generate outputs
|
20 |
+
- Feed the chat template to DeciLM-7b-instruct quantized to INT8.
|
21 |
+
```text
|
22 |
+
./main -m ~/Downloads/decilm-7b-uniform-gqa-q8_0.gguf -p """
|
23 |
+
### System:
|
24 |
+
You are an AI assistant that follows instruction extremely well. Help as much as you can.
|
25 |
+
### User:
|
26 |
+
How do I make the most delicious pancakes the world has ever tasted?
|
27 |
+
### Assistant:
|
28 |
+
"""
|
29 |
+
```
|
30 |
+
|
31 |
+
- Running on MacBook M2 Pro 32gb::
|
32 |
+
```
|
33 |
+
### System:
|
34 |
+
You are an AI assistant that follows instruction extremely well. Help as much as you can.
|
35 |
+
### User:
|
36 |
+
How do I make the most delicious pancakes the world has ever tasted?
|
37 |
+
### Assistant:
|
38 |
+
To make the most delicious pancakes the world ever tasted, follow these steps:
|
39 |
+
|
40 |
+
1. In a mixing bowl, combine 2 cups of all-purpose flour, 4 tablespoons of sugar, and 3 teaspoon of baking powder with 1/2 teaspoon salt, mix well.
|
41 |
+
2. Make a hole in the center and pour in 4 eggs and 1 cup of milk, whisk well mix it until smooth. Add 3 table spoon of oil and a tables of melted butter.
|
42 |
+
3. Heat your frying pan with little bit butter or oil and ladle batter onto the pan, spread it with 1/2 inch width. Wait for small bubbles to form in the surface and flip over to brown other side until golden.
|
43 |
+
4. Enjoy your delicious pancakes [end of text]
|
44 |
+
|
45 |
+
llama_print_timings: load time = 343.16 ms
|
46 |
+
llama_print_timings: sample time = 14.69 ms / 172 runs ( 0.09 ms per token, 11712.63 tokens per second)
|
47 |
+
llama_print_timings: prompt eval time = 239.48 ms / 52 tokens ( 4.61 ms per token, 217.14 tokens per second)
|
48 |
+
llama_print_timings: eval time = 7767.20 ms / 171 runs ( 45.42 ms per token, 22.02 tokens per second)
|
49 |
+
llama_print_timings: total time = 8045.89 ms
|
50 |
+
ggml_metal_free: deallocating
|
51 |
+
Log end
|
52 |
+
```
|