neuralmagic/Meta-Llama-3.1-405B-Instruct-quantized.w8a16 Text Generation • Updated Oct 9, 2024 • 191 • 2
neuralmagic/Meta-Llama-3.1-405B-Instruct-quantized.w8a8 Text Generation • Updated Dec 3, 2024 • 184 • 2
Research projects on top of vLLM Collection Papers cited in https://blog.vllm.ai/2024/07/25/lfai-perf.html • 6 items • Updated Jul 29, 2024 • 12
Efficient Memory Management for Large Language Model Serving with PagedAttention Paper • 2309.06180 • Published Sep 12, 2023 • 25
Running on CPU Upgrade 12.8k 12.8k Open LLM Leaderboard 🏆 Track, rank and evaluate open LLMs and chatbots