Explore, Curate and Vector Search Any Hugging Face Dataset with Nomic Atlas
•
26
Pedro Cuenca
pcuenq
AI & ML interests
None yet
Recent Activity
liked
a model
2 days ago
deepseek-ai/DeepSeek-R1
upvoted
an
article
2 days ago
Explore, Curate and Vector Search Any Hugging Face Dataset with Nomic Atlas
upvoted
an
article
3 days ago
Mastering Long Contexts in LLMs with KVPress
Articles
Organizations
Posts
1
Post
4940
OpenELM in Core ML
Apple recently released a set of efficient LLMs in sizes varying between 270M and 3B parameters. Their quality, according to benchmarks, is similar to OLMo models of comparable size, but they required half the pre-training tokens because they use layer-wise scaling, where the number of attention heads increases in deeper layers.
I converted these models to Core ML, for use on Apple Silicon, using this script: https://gist.github.com/pcuenca/23cd08443460bc90854e2a6f0f575084. The converted models were uploaded to this community in the Hub for anyone that wants to integrate inside their apps: corenet-community/openelm-core-ml-6630c6b19268a5d878cfd194
The conversion was done with the following parameters:
- Precision: float32.
- Sequence length: fixed to 128.
With swift-transformers (https://github.com/huggingface/swift-transformers), I'm getting about 56 tok/s with the 270M on my M1 Max, and 6.5 with the largest 3B model. These speeds could be improved by converting to
I'm also looking at optimizing inference using an experimental kv cache in swift-transformers. It's a bit tricky because the layers have varying number of attention heads, but I'm curious to see how much this feature can accelerate performance in this model family :)
Regarding the instruct fine-tuned models, I don't know the chat template that was used. The models use the Llama 2 tokenizer, but the Llama 2 chat template, or the default Alignment Handbook one that was used to train, are not recognized. Any ideas on this welcome!
Apple recently released a set of efficient LLMs in sizes varying between 270M and 3B parameters. Their quality, according to benchmarks, is similar to OLMo models of comparable size, but they required half the pre-training tokens because they use layer-wise scaling, where the number of attention heads increases in deeper layers.
I converted these models to Core ML, for use on Apple Silicon, using this script: https://gist.github.com/pcuenca/23cd08443460bc90854e2a6f0f575084. The converted models were uploaded to this community in the Hub for anyone that wants to integrate inside their apps: corenet-community/openelm-core-ml-6630c6b19268a5d878cfd194
The conversion was done with the following parameters:
- Precision: float32.
- Sequence length: fixed to 128.
With swift-transformers (https://github.com/huggingface/swift-transformers), I'm getting about 56 tok/s with the 270M on my M1 Max, and 6.5 with the largest 3B model. These speeds could be improved by converting to
float16. However, there's some precision loss somewhere and generation doesn't work in float16 mode yet. I'm looking into this and will keep you posted! Or take a look at this issue if you'd like to help: https://github.com/huggingface/swift-transformers/issues/95I'm also looking at optimizing inference using an experimental kv cache in swift-transformers. It's a bit tricky because the layers have varying number of attention heads, but I'm curious to see how much this feature can accelerate performance in this model family :)
Regarding the instruct fine-tuned models, I don't know the chat template that was used. The models use the Llama 2 tokenizer, but the Llama 2 chat template, or the default Alignment Handbook one that was used to train, are not recognized. Any ideas on this welcome!
Collections
4
spaces
10
models
103
pcuenq/vit_large_patch14_dinov2.lvd142m
Image Feature Extraction
•
Updated
•
8
pcuenq/vit_base_patch32_clip_448.laion2b_ft_in12k_in1k
Image Classification
•
Updated
•
12
pcuenq/tw1
Image Classification
•
Updated
•
4
pcuenq/timm-wrapper-food-lora
Updated
pcuenq/vit_base_patch16_224.augreg2_in21k_ft_in1k.lora_ft_food101
Updated
pcuenq/distilbert-base-multilingual-cased-tokenizer
Fill-Mask
•
Updated
•
20
pcuenq/convnext_nano.r384_ad_in12k
Image Classification
•
Updated
•
5
pcuenq/gatingtests
Updated
pcuenq/gemma-2-2b-it-float32
Updated
pcuenq/gemma-2-2b-it-4bit-test
Text Generation
•
Updated
•
10
datasets
17
pcuenq/diffusers-videos
Viewer
•
Updated
•
4
•
9
pcuenq/coco-semantic-segmentation
Viewer
•
Updated
•
5k
•
108
pcuenq/misc-assets
Updated
•
26
pcuenq/media
Updated
•
2
pcuenq/tests
Viewer
•
Updated
•
27
•
127
pcuenq/amused_mps
Updated
•
18
pcuenq/gists
Updated
•
15
•
6
pcuenq/recipe-images
Viewer
•
Updated
•
8
•
18
pcuenq/face_synthetics_spiga
Viewer
•
Updated
•
100k
•
470
•
13
pcuenq/face_synthetics_spiga_smol
Viewer
•
Updated
•
100
•
42