avideci commited on
Commit
5b748ad
1 Parent(s): 2ff83fb

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +43 -32
README.md CHANGED
@@ -2,40 +2,51 @@
2
  license: apache-2.0
3
  ---
4
 
5
- # DeciLM-7b-instruct GGUF checkpoints (llama.cpp compatible)
 
 
6
 
7
-
8
- ## Chat template
9
- - Feed the chat template to DeciLM-7b-instruct quantized to INT8.
10
- ```text
11
- ./main -m ~/Downloads/decilm-7b-uniform-gqa-q8_0.gguf -p """
12
- ### System:
13
- You are an AI assistant that follows instruction extremely well. Help as much as you can.
14
- ### User:
15
- How do I make the most delicious pancakes the world has ever tasted?
16
- ### Assistant:
17
- """
18
  ```
19
-
20
- - Running on MacBook M2 Pro 32gb::
21
  ```
22
- ### System:
23
- You are an AI assistant that follows instruction extremely well. Help as much as you can.
24
- ### User:
25
- How do I make the most delicious pancakes the world has ever tasted?
26
- ### Assistant:
27
- To make the most delicious pancakes the world ever tasted, follow these steps:
28
 
29
- 1. In a mixing bowl, combine 2 cups of all-purpose flour, 4 tablespoons of sugar, and 3 teaspoon of baking powder with 1/2 teaspoon salt, mix well.
30
- 2. Make a hole in the center and pour in 4 eggs and 1 cup of milk, whisk well mix it until smooth. Add 3 table spoon of oil and a tables of melted butter.
31
- 3. Heat your frying pan with little bit butter or oil and ladle batter onto the pan, spread it with 1/2 inch width. Wait for small bubbles to form in the surface and flip over to brown other side until golden.
32
- 4. Enjoy your delicious pancakes [end of text]
33
 
34
- llama_print_timings: load time = 343.16 ms
35
- llama_print_timings: sample time = 14.69 ms / 172 runs ( 0.09 ms per token, 11712.63 tokens per second)
36
- llama_print_timings: prompt eval time = 239.48 ms / 52 tokens ( 4.61 ms per token, 217.14 tokens per second)
37
- llama_print_timings: eval time = 7767.20 ms / 171 runs ( 45.42 ms per token, 22.02 tokens per second)
38
- llama_print_timings: total time = 8045.89 ms
39
- ggml_metal_free: deallocating
40
- Log end
41
- ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
2
  license: apache-2.0
3
  ---
4
 
5
+ # DeciLM-7b-instruct GGUF checkpoints
6
+ This repository includes DeciLM-7b-instruct checkpoints in the GGUF format.<br>
7
+ DeciLM performs well on commodity CPUs using the llama.cpp codebase.
8
 
9
+ ## 1. Clone and build llama.cpp (1 minute)
 
 
 
 
 
 
 
 
 
 
10
  ```
11
+ git clone https://github.com/ggerganov/llama.cpp && cd llama.cpp && make -j
 
12
  ```
 
 
 
 
 
 
13
 
14
+ ## 2. Download the GGUF checkpoint
15
+ - Go to "Files"
16
+ - Click on "decilm-7b-uniform-gqa-q8_0.gguf"
17
+ - Click on the "Download" button
18
 
19
+ ## 3. Generate outputs
20
+ - Feed the chat template to DeciLM-7b-instruct quantized to INT8.
21
+ ```text
22
+ ./main -m ~/Downloads/decilm-7b-uniform-gqa-q8_0.gguf -p """
23
+ ### System:
24
+ You are an AI assistant that follows instruction extremely well. Help as much as you can.
25
+ ### User:
26
+ How do I make the most delicious pancakes the world has ever tasted?
27
+ ### Assistant:
28
+ """
29
+ ```
30
+
31
+ - Running on MacBook M2 Pro 32gb::
32
+ ```
33
+ ### System:
34
+ You are an AI assistant that follows instruction extremely well. Help as much as you can.
35
+ ### User:
36
+ How do I make the most delicious pancakes the world has ever tasted?
37
+ ### Assistant:
38
+ To make the most delicious pancakes the world ever tasted, follow these steps:
39
+
40
+ 1. In a mixing bowl, combine 2 cups of all-purpose flour, 4 tablespoons of sugar, and 3 teaspoon of baking powder with 1/2 teaspoon salt, mix well.
41
+ 2. Make a hole in the center and pour in 4 eggs and 1 cup of milk, whisk well mix it until smooth. Add 3 table spoon of oil and a tables of melted butter.
42
+ 3. Heat your frying pan with little bit butter or oil and ladle batter onto the pan, spread it with 1/2 inch width. Wait for small bubbles to form in the surface and flip over to brown other side until golden.
43
+ 4. Enjoy your delicious pancakes [end of text]
44
+
45
+ llama_print_timings: load time = 343.16 ms
46
+ llama_print_timings: sample time = 14.69 ms / 172 runs ( 0.09 ms per token, 11712.63 tokens per second)
47
+ llama_print_timings: prompt eval time = 239.48 ms / 52 tokens ( 4.61 ms per token, 217.14 tokens per second)
48
+ llama_print_timings: eval time = 7767.20 ms / 171 runs ( 45.42 ms per token, 22.02 tokens per second)
49
+ llama_print_timings: total time = 8045.89 ms
50
+ ggml_metal_free: deallocating
51
+ Log end
52
+ ```