hugging-quants

/

Meta-Llama-3.1-405B-Instruct-GPTQ-INT4

like 16

/usr4/ec523/brucejia/.local/lib/python3.10/site-packages/accelerate/utils/modeling.py:1405: UserWarning: Current model requires 45201611776 bytes of buffer for offloaded layers, which seems does not fit any GPU's remaining memory. If you are experiencing a OOM later, please consider using offload_buffers=True.