Text Generation
Transformers
GGUF
English
Inference Endpoints
leafspark commited on
Commit
0aefaec
1 Parent(s): 9cd9460

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +5 -3
README.md CHANGED
@@ -20,11 +20,13 @@ The GGUFs uploaded are full FP32 precision.
20
  Using OpenOrca GPT-4 data + cosmopedia for some extra data + dolly15k for instruct
21
 
22
  ## Model Details:
23
- - 40M parameters
24
  - 8 attention heads
25
- - 32 layers
26
  - 384 embeddings size
27
- - 2048/8192 context (please use 4x RoPE scaling)
 
 
28
 
29
  ## Prompt Format (Alpaca):
30
  ```
 
20
  Using OpenOrca GPT-4 data + cosmopedia for some extra data + dolly15k for instruct
21
 
22
  ## Model Details:
23
+ - 71.7M parameters (71,775,700)
24
  - 8 attention heads
25
+ - 32 layers (34 layers on final model)
26
  - 384 embeddings size
27
+ - 2048/8192/16384 context (please use 4x RoPE scaling, may train a 16k finetuned version later)
28
+ - Batch size 16
29
+ - llama.cpp (train-text-from-scratch)
30
 
31
  ## Prompt Format (Alpaca):
32
  ```