Spaces:
Running
on
Zero
Memory limit exceeded on A10G Large (46GB RAM)
Hello! I'm trying to run this Space on upgraded hardware. I first tried the A10G Small and got a 14Gi memory error. I have now upgraded to the A10G Large, but I'm still getting a Memory limit exceeded (46Gi) error during application startup, seemingly while it's loading the model shards.
Is there a specific hardware configuration needed to run this, or is this a known issue? Thank you!
Our Hugging Face Space does require substantial RAM to run, approaching the 69GB limit of our Space settings. We’ve modified our code to load models directly in bfloat16 format, which reduces peak RAM usage—we hope this helps.
Additionally, I’ve noted that a single A10G GPU (with 24GB VRAM) cannot smoothly execute our current implementation. We recommend using our GitHub version instead, where we’ve recently added VRAM and RAM optimizations that significantly reduce memory requirements. For details, please refer to our GitHub repository.