Text Generation
Transformers
PyTorch
GGUF
English
llama
text-generation-inference
PY007 commited on
Commit
8f37d86
·
1 Parent(s): def7ba2

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +4 -1
README.md CHANGED
@@ -22,8 +22,11 @@ The TinyLlama project aims to **pretrain** a **1.1B Llama model on 3 trillion to
22
  We adopted exactly the same architecture and tokenizer as Llama 2. This means TinyLlama can be plugged and played in many open-source projects built upon Llama. Besides, TinyLlama is compact with only 1.1B parameters. This compactness allows it to cater to a multitude of applications demanding a restricted computation and memory footprint.
23
 
24
  #### This Model
25
- This is an intermediate checkpoint with 240K steps and 503B tokens. **We suggest you not use this directly for inference.** The [chat model](https://huggingface.co/PY007/TinyLlama-1.1B-Chat-v0.1) is always preferred **
26
 
 
 
 
27
 
28
  #### How to use
29
  You will need the transformers>=4.31
 
22
  We adopted exactly the same architecture and tokenizer as Llama 2. This means TinyLlama can be plugged and played in many open-source projects built upon Llama. Besides, TinyLlama is compact with only 1.1B parameters. This compactness allows it to cater to a multitude of applications demanding a restricted computation and memory footprint.
23
 
24
  #### This Model
25
+ This is a code LM finetuned(or so-called continue pretrianed) from the 500B TinyLlama checkpoint with another 7B Python data from the starcoderdata.
26
 
27
+ While the finetuning data is exclusively Python, the model retains its ability in many other languages such as C or Java.
28
+
29
+ The HumanEval accuracy is **14**.
30
 
31
  #### How to use
32
  You will need the transformers>=4.31