有出量化模型INT4的计划吗

#21

by Winkuis - opened Jun 13, 2024

Jun 13, 2024

有出量化模型INT4的计划吗，非量化太大了，机器跑不动，想尝试一下

Knowledge Engineering Group (KEG) & Data Mining at Tsinghua University org Jun 15, 2024

我们有提供load in 4bit，如果你的内存没办法做到以load in 4bit读入，魔搭社区有社区直接把load in 4bit 导出的4bit模型

zRzRzRzRzRzRzR changed discussion status to closed Jun 15, 2024

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment