Update README.md
Browse files
README.md
CHANGED
@@ -1,12 +1,16 @@
|
|
1 |
---
|
2 |
license: apache-2.0
|
|
|
|
|
3 |
---
|
4 |
|
5 |
-
# llama.cpp compatible
|
6 |
-
## Usage with chat template
|
7 |
|
|
|
|
|
|
|
8 |
```text
|
9 |
-
./main -m
|
10 |
### System:
|
11 |
You are an AI assistant that follows instruction extremely well. Help as much as you can.
|
12 |
### User:
|
@@ -15,7 +19,7 @@ How do I make the most delicious pancakes the world has ever tasted?
|
|
15 |
"""
|
16 |
```
|
17 |
|
18 |
-
|
19 |
```
|
20 |
### System:
|
21 |
You are an AI assistant that follows instruction extremely well. Help as much as you can.
|
|
|
1 |
---
|
2 |
license: apache-2.0
|
3 |
+
language:
|
4 |
+
- ba
|
5 |
---
|
6 |
|
7 |
+
# DeciLM-7b-instruct GGUF checkpoints (llama.cpp compatible)
|
|
|
8 |
|
9 |
+
|
10 |
+
## Chat template
|
11 |
+
- Feed the chat template to DeciLM-7b-instruct quantized to INT8.
|
12 |
```text
|
13 |
+
./main -m ~/Downloads/decilm-7b-uniform-gqa-q8_0.gguf -p """
|
14 |
### System:
|
15 |
You are an AI assistant that follows instruction extremely well. Help as much as you can.
|
16 |
### User:
|
|
|
19 |
"""
|
20 |
```
|
21 |
|
22 |
+
- Running on MacBook M2 Pro 32gb::
|
23 |
```
|
24 |
### System:
|
25 |
You are an AI assistant that follows instruction extremely well. Help as much as you can.
|