GGUF
English
Inference Endpoints
mav23 commited on
Commit
4bf452e
•
1 Parent(s): 9053886

Upload folder using huggingface_hub

Browse files
.gitattributes CHANGED
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ tinyllama-1.1b-chat-v0.1.Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,61 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ datasets:
4
+ - cerebras/SlimPajama-627B
5
+ - bigcode/starcoderdata
6
+ - timdettmers/openassistant-guanaco
7
+ language:
8
+ - en
9
+ ---
10
+ <div align="center">
11
+
12
+ # TinyLlama-1.1B
13
+ </div>
14
+
15
+ https://github.com/jzhang38/TinyLlama
16
+
17
+ The TinyLlama project aims to **pretrain** a **1.1B Llama model on 3 trillion tokens**. With some proper optimization, we can achieve this within a span of "just" 90 days using 16 A100-40G GPUs 🚀🚀. The training has started on 2023-09-01.
18
+
19
+ <div align="center">
20
+ <img src="./TinyLlama_logo.png" width="300"/>
21
+ </div>
22
+
23
+ We adopted exactly the same architecture and tokenizer as Llama 2. This means TinyLlama can be plugged and played in many open-source projects built upon Llama. Besides, TinyLlama is compact with only 1.1B parameters. This compactness allows it to cater to a multitude of applications demanding a restricted computation and memory footprint.
24
+
25
+ #### This Model
26
+ This is the chat model finetuned on [PY007/TinyLlama-1.1B-intermediate-step-240k-503b](https://huggingface.co/PY007/TinyLlama-1.1B-intermediate-step-240k-503b). The dataset used is [openassistant-guananco](https://huggingface.co/datasets/timdettmers/openassistant-guanaco).
27
+
28
+ #### How to use
29
+ You will need the transformers>=4.31
30
+ Do check the [TinyLlama](https://github.com/jzhang38/TinyLlama) github page for more information.
31
+ ```python
32
+ from transformers import AutoTokenizer
33
+ import transformers
34
+ import torch
35
+ model = "PY007/TinyLlama-1.1B-Chat-v0.1"
36
+ tokenizer = AutoTokenizer.from_pretrained(model)
37
+ pipeline = transformers.pipeline(
38
+ "text-generation",
39
+ model=model,
40
+ torch_dtype=torch.float16,
41
+ device_map="auto",
42
+ )
43
+
44
+ prompt = "What are the values in open source projects?"
45
+ formatted_prompt = (
46
+ f"### Human: {prompt}### Assistant:"
47
+ )
48
+
49
+
50
+ sequences = pipeline(
51
+ formatted_prompt,
52
+ do_sample=True,
53
+ top_k=50,
54
+ top_p = 0.7,
55
+ num_return_sequences=1,
56
+ repetition_penalty=1.1,
57
+ max_new_tokens=500,
58
+ )
59
+ for seq in sequences:
60
+ print(f"Result: {seq['generated_text']}")
61
+ ```
tinyllama-1.1b-chat-v0.1.Q4_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:41f48e59f22abe7b283c42d4799f8c50b0b96d3fca11a6c29644dac32e575183
3
+ size 636729216