@csabakecskemeti on Hugging Face: "The AMD Instinct MI50 (~$110) is surprisingly fast for inference Quantized…"

Join the conversation

Join the community of Machine Learners and AI enthusiasts.

csabakecskemeti

posted an update 10 days ago

Post

4460

The AMD Instinct MI50 (~$110) is surprisingly fast for inference Quantized models.

This runs a Llama 3.1 8B Q8 with Llama.cpp
DevQuasar/Mi50

A little blogpost about the HW
http://devquasar.com/uncategorized/amd-radeon-instinct-mi50-cheap-inference/

In this post

csabakecskemeti Csaba Kecskemeti