pszemraj
/

Llama-3-6.3b-v0.1

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

pszemraj commited on May 18

Commit

851c3cd

•

1 Parent(s): da2bc52

Update README.md

Files changed (1) hide show

README.md +2 -2

README.md CHANGED Viewed

@@ -14,11 +14,11 @@ language:
 This is a layer pruning experiment based off of the original llama-3-8b:
-- 8 layers pruned with PruneMe/MergeKit
   - layers selected using [BEE-spoke-data/fineweb-100k_en-med](https://hf.co/datasets/BEE-spoke-data/fineweb-100k_en-med)
 - brief subsequent continued pretraining @ ctx 4096
   - data: 10k rows of FineWeb (different than pruning data) + some curated data
 ## quick eval

 This is a layer pruning experiment based off of the original llama-3-8b:
+- 8 layers pruned with [PruneMe](https://github.com/pszemraj/PruneMe/tree/upgrades)/MergeKit
   - layers selected using [BEE-spoke-data/fineweb-100k_en-med](https://hf.co/datasets/BEE-spoke-data/fineweb-100k_en-med)
 - brief subsequent continued pretraining @ ctx 4096
   - data: 10k rows of FineWeb (different than pruning data) + some curated data
+- wandb [here](https://wandb.ai/pszemraj/llama3-pruning)
 ## quick eval