# DeepSeek-R1-Distill-Qwen-7B-q4f16_ft-MLC

|                     |                                             Model Configuration                                             |
|---------------------|:-----------------------------------------------------------------------------------------------------------:|
| Source Model        | [`deepseek-ai/DeepSeek-R1-Distill-Qwen-7B`](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-7B) |
| Inference API       |                                                  `MLC_LLM`                                                  |
| Quantization        |                                                 `q4f16_ft`                                                  |
| Model Type          |                                                   `qwen2`                                                   |
| Vocab Size          |                                                  `152064`                                                   |
| Context Window Size |                                                  `131072`                                                   |
| Prefill Chunk Size  |                                                   `8192`                                                    |
| Temperature         |                                                    `0.6`                                                    |
| Repetition Penalty  |                                                    `1.0`                                                    |
| top_p               |                                                   `0.95`                                                    |
| pad_token_id        |                                                     `0`                                                     |
| bos_token_id        |                                                  `151646`                                                   |
| eos_token_id        |                                                  `151643`                                                   |

See [`jetson-ai-lab.com/models.html`](https://jetson-ai-lab.com/models.html) for benchmarks, examples, and containers to deploy local serving and inference for these quantized models.