Post
2600
I got 370 tokens/sec of Qwen3-30B-A3B 2507 on my desktop Z8 GPU workstation. My target is 400 t/s, and the last 10 % always tastes like victory!
- 3 replies
spooner
spooner2
AI & ML interests
None yet
Recent Activity
liked
a model
3 months ago
ubergarm/DeepSeek-R1-0528-GGUF
Organizations
None yet
reacted to
mitkox's
post with 🔥
17 days ago
reacted to
eaddario's
post with 🚀
about 2 months ago
Post
3734
Layer-wise and Pruned versions of Qwen/Qwen3-30B-A3B
* Tesor-wise: eaddario/Qwen3-30B-A3B-GGUF
* Pruned: eaddario/Qwen3-30B-A3B-pruned-GGUF
Even though the Perplexity scores of the pruned version are 3 times higher, the ARC, HellaSwag, MMLU, Truthful QA and WinoGrande scores are holding remarkably well, considering two layers were removed (5 and 39). This seems to support Xin Men et al conclusions in
ShortGPT: Layers in Large Language Models are More Redundant Than You Expect (2403.03853)
Results summary in the model's card and test results in the ./scores directory. Questions/feedback is always welcomed.
* Tesor-wise: eaddario/Qwen3-30B-A3B-GGUF
* Pruned: eaddario/Qwen3-30B-A3B-pruned-GGUF
Even though the Perplexity scores of the pruned version are 3 times higher, the ARC, HellaSwag, MMLU, Truthful QA and WinoGrande scores are holding remarkably well, considering two layers were removed (5 and 39). This seems to support Xin Men et al conclusions in
ShortGPT: Layers in Large Language Models are More Redundant Than You Expect (2403.03853)
Results summary in the model's card and test results in the ./scores directory. Questions/feedback is always welcomed.
liked
a
model
3 months ago
liked
3
models
4 months ago
FIXED: Failed to parse Jinja template
3
#2 opened 4 months ago
by
wapxmas
reacted to
Xenova's
post with 🔥
4 months ago
Post
8348
Introducing the ONNX model explorer: Browse, search, and visualize neural networks directly in your browser. 🤯 A great tool for anyone studying Machine Learning! We're also releasing the entire dataset of graphs so you can use them in your own projects! 🤗
Check it out! 👇
Demo: onnx-community/model-explorer
Dataset: onnx-community/model-explorer
Source code: https://github.com/xenova/model-explorer
Check it out! 👇
Demo: onnx-community/model-explorer
Dataset: onnx-community/model-explorer
Source code: https://github.com/xenova/model-explorer
upvoted
a
collection
4 months ago
Error HyVideoModelLoader
14
#7 opened 6 months ago
by
Nikita661995
liked
a
model
7 months ago
upvoted
a
paper
10 months ago
reacted to
yongchanghao's
post with 🔥
10 months ago
Post
3796
We just released a paper (NeuZip) that compresses VRAM in a lossless manner to run larger models. This should be particularly useful when VRAM is insufficient during training/inference. Specifically, we look inside each floating number and find that the exponents are highly compressible (as shown in the figure below).
Read more about the work at NeuZip: Memory-Efficient Training and Inference with Dynamic Compression of Neural Networks (2410.20650)
Read more about the work at NeuZip: Memory-Efficient Training and Inference with Dynamic Compression of Neural Networks (2410.20650)
liked
a
model
12 months ago
combined safetensors , but comfyui issue a error.
9
#3 opened about 1 year ago
by
demo001s
liked
3
models
about 1 year ago
liked
a
model
over 1 year ago
reacted to
codelion's
post with 🚀
over 1 year ago
Post
1765
Happy to announce the open source framework to turbo charge devops called patchwork - https://github.com/patched-codes/patchwork
You can use it to build patchflows - workflows that use LLMs for software development tasks like bug fixing, pull request review, library migration and documentation.
Supports any LLM of your choice including our own MoE model - patched-codes/patched-mix-4x7B
Give it a try!
You can use it to build patchflows - workflows that use LLMs for software development tasks like bug fixing, pull request review, library migration and documentation.
Supports any LLM of your choice including our own MoE model - patched-codes/patched-mix-4x7B
Give it a try!
- 2 replies