Salesforce

/

xgen-7b-8k-base

like 318

Follow

Salesforce 698

KeyError Traceback (most recent call last)
Cell In [4], line 1
----> 1 model = AutoModelForCausalLM.from_pretrained("Salesforce/xgen-7b-8k-base", torch_dtype=torch.bfloat16).to("cuda")

File /usr/local/lib/python3.9/dist-packages/transformers/models/auto/auto_factory.py:434, in _BaseAutoModelClass.from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
432 hub_kwargs = {name: kwargs.pop(name) for name in hub_kwargs_names if name in kwargs}
433 if not isinstance(config, PretrainedConfig):
--> 434 config, kwargs = AutoConfig.from_pretrained(
435 pretrained_model_name_or_path,
436 return_unused_kwargs=True,
437 trust_remote_code=trust_remote_code,
438 **hub_kwargs,
439 **kwargs,
440 )
441 if hasattr(config, "auto_map") and cls.name in config.auto_map:
442 if not trust_remote_code:

File /usr/local/lib/python3.9/dist-packages/transformers/models/auto/configuration_auto.py:829, in AutoConfig.from_pretrained(cls, pretrained_model_name_or_path, **kwargs)
827 return config_class.from_pretrained(pretrained_model_name_or_path, **kwargs)
828 elif "model_type" in config_dict:
--> 829 config_class = CONFIG_MAPPING[config_dict["model_type"]]
830 return config_class.from_dict(config_dict, **unused_kwargs)
831 else:
832 # Fallback: use pattern matching on the string.
833 # We go from longer names to shorter names to catch roberta before bert (for instance)

File /usr/local/lib/python3.9/dist-packages/transformers/models/auto/configuration_auto.py:536, in _LazyConfigMapping.getitem(self, key)
534 return self._extra_content[key]
535 if key not in self._mapping:
--> 536 raise KeyError(key)
537 value = self._mapping[key]
538 module_name = model_type_to_module_name(key)

KeyError: 'llama'

Hi, I have used the exact same commands as in the readme, however, I am facing the above error in the line:
model = AutoModelForCausalLM.from_pretrained("Salesforce/xgen-7b-8k-base", torch_dtype=torch.bfloat16).to("cuda")

It is able to download config.json though!