A version of https://huggingface.co/xai-org/grok-1 that is compatible with the latest SGLang.
The tokenizer is from https://huggingface.co/Xenova/grok-1-tokenizer
# Load real weights
python3 -m sglang.launch_server --model lmzheng/grok-1 --tp 8 --quant fp8
# Load dummy weights
python3 -m sglang.launch_server --model lmzheng/grok-1 --tp 8 --quant fp8 --load-format dummy
# Test accuracy on gsm-8k (sanity check), it should be around 0.75
python3 -m sglang.test.few_shot_gsm8k --num-questions 200
- Downloads last month
- 3,629