TypeError: RefinedWebModel isn't supported yet.
This is what my code is
from transformers import AutoTokenizer
from auto_gptq import AutoGPTQForCausalLM
# Download the model from HF and store it locally, then reference its location here:
#quantized_model_dir = model_path
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("TheBloke/WizardLM-Uncensored-Falcon-7B-GPTQ", use_fast=False)
model = AutoGPTQForCausalLM.from_quantized("TheBloke/WizardLM-Uncensored-Falcon-7B-GPTQ", device="cuda:1", use_triton=False, use_safetensors=True, trust_remote_code=True)
When I Run This, i Got These:
- configuration_RW.py
. Make sure to double-check they do not contain any added malicious code. To avoid downloading new versions of the code file, you can pin a revision.
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ in <cell line: 11>:11 │
│ │
│ /usr/local/lib/python3.10/dist-packages/auto_gptq/modeling/auto.py:62 in from_quantized │
│ │
│ 59 │ │ model_basename: Optional[str] = None, │
│ 60 │ │ trust_remote_code: bool = False │
│ 61 │ ) -> BaseGPTQForCausalLM: │
│ ❱ 62 │ │ model_type = check_and_get_model_type(save_dir) │
│ 63 │ │ return GPTQ_CAUSAL_LM_MODEL_MAP[model_type].from_quantized( │
│ 64 │ │ │ save_dir=save_dir, │
│ 65 │ │ │ device=device, │
│ │
│ /usr/local/lib/python3.10/dist-packages/auto_gptq/modeling/_utils.py:124 in │
│ check_and_get_model_type │
│ │
│ 121 def check_and_get_model_type(model_dir): │
│ 122 │ config = AutoConfig.from_pretrained(model_dir, trust_remote_code=True) │
│ 123 │ if config.model_type not in SUPPORTED_MODELS: │
│ ❱ 124 │ │ raise TypeError(f"{config.model_type} isn't supported yet.") │
│ 125 │ model_type = config.model_type │
│ 126 │ return model_type │
│ 127 │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
TypeError: RefinedWebModel isn't supported yet.
How Should I Get Rid of These?
Can SomeBody Help?
@TheBloke can you please chime in. Thanks!
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ in <cell line: 11>:11 │
│ │
│ /usr/local/lib/python3.10/dist-packages/auto_gptq/modeling/auto.py:62 in from_quantized │
│ │
│ 59 │ │ model_basename: Optional[str] = None, │
│ 60 │ │ trust_remote_code: bool = False │
│ 61 │ ) -> BaseGPTQForCausalLM: │
│ ❱ 62 │ │ model_type = check_and_get_model_type(save_dir) │
│ 63 │ │ return GPTQ_CAUSAL_LM_MODEL_MAP[model_type].from_quantized( │
│ 64 │ │ │ save_dir=save_dir, │
│ 65 │ │ │ device=device, │
│ │
│ /usr/local/lib/python3.10/dist-packages/auto_gptq/modeling/_utils.py:124 in │
│ check_and_get_model_type │
│ │
│ 121 def check_and_get_model_type(model_dir): │
│ 122 │ config = AutoConfig.from_pretrained(model_dir, trust_remote_code=True) │
│ 123 │ if config.model_type not in SUPPORTED_MODELS: │
│ ❱ 124 │ │ raise TypeError(f"{config.model_type} isn't supported yet.") │
│ 125 │ model_type = config.model_type │
│ 126 │ return model_type │
│ 127 │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
TypeError: RefinedWebModel isn't supported yet.
Did You Found Any Solution, My Friend?
You need to be using auto-gptq version 0.2.0. For some reason you guys don't have the right version.
Please try:
pip uninstall auto-gptq
pip install auto-gptq==0.2.0
I Showed The Log Here: https://huggingface.co/TheBloke/falcon-7b-instruct-GPTQ/discussions/9
You need to be using auto-gptq version 0.2.0. For some reason you guys don't have the right version.
Please try:
pip uninstall auto-gptq pip install auto-gptq==0.2.0
I donno somehow it says it does'nt have version 0.2.0
thank you @TheBloke
this works for me in colab google
!git clone https://github.com/PanQiWei/AutoGPTQ
%cd AutoGPTQ
!pip install .
and then download in colab as per below sequence.
- !huggingface-cli login --token "hf_xxxxxxxxx"
- from huggingface_hub import snapshot_download
snapshot_download(repo_id="TheBloke/WizardLM-Uncensored-Falcon-7B-GPTQ")
this wil dowdload to the colab instance root folder (this folder will be shown after executing step 2 above)
the rest are the same. however i received some warning notifications but it still works.
Yeah currently a lot of warnings are printed, stuff like:
WARNING:RWGPTQForCausalLM hasn't fused attention module yet, will skip inject fused attention.
WARNING:RWGPTQForCausalLM hasn't fused mlp module yet, will skip inject fused mlp.
This can be ignored. I will suggest to the AutoGPTQ author that these warnings should be INFO instead, or not printed
The only working "old" version of auto-gptq at the moment is %pip install auto-gptq==0.2.2 on colab.