allenai

/

OLMo-2-0325-32B

like 19

Follow

Ai2 2.82k

Description

When launching the Hugging Face server with the model allenai/OLMo-2-0325-32B, an error occurs related to an unexpected hidden_size value in the model's execution. The error traceback indicates a mismatch in expected and found hidden size values.

Command Used

python3 -m huggingfaceserver --model_id=allenai/OLMo-2-0325-32B

Logs (Relevant Excerpt)

ValueError: Expected hidden_size to be 5120, but found: 2390

Full traceback:

2025-03-14 07:47:29.428 uvicorn.error ERROR:    Traceback (most recent call last):
  ... (truncated for brevity) ...
  File "/prod_venv/lib/python3.10/site-packages/vllm/model_executor/layers/layernorm.py", line 52, in forward_native
    raise ValueError("Expected hidden_size to be 5120, but found: 2390")
ValueError: Expected hidden_size to be 5120, but found: 2390

Expected Behavior

The model should load and execute without encountering a hidden size mismatch.

Environment Details

• Model: allenai/OLMo-2-0325-32B
• Torch Version: torch.float16
• Platform: CUDA
• Server: huggingfaceserver
• Backend: vLLM

Steps to Reproduce

1. Run the command provided above.
2. Observe the model loading process.
3. Encounter the ValueError when executing the model.

Possible Causes

• Mismatch in expected model configuration (hidden_size values inconsistency).
• Incorrect model loading behavior when using safetensors.

Additional Notes

Would appreciate any insights into why the hidden_size value is different from the expected 5120.
I also tried with the allenai/OLMo-2-0325-32B-Instruct model and I got the same error.

Thank you!