Glavin001 commited on
Commit
7dd61bc
·
1 Parent(s): f0c0909

Add missing pad_token_id config property

Browse files

Fix error:

handler: 'pad_token_id'
traceback: Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/runpod/serverless/modules/job.py", line 141, in run_job_generator
for output_partial in job_output:
File "/data/handler.py", line 107, in inference
generator, default_settings = load_model()
File "/data/handler.py", line 45, in load_model
config = ExLlamaConfig(model_config_path) # create config from config.json
File "/data/exllama/model.py", line 52, in __init__
self.pad_token_id = read_config["pad_token_id"]
KeyError: 'pad_token_id'

Files changed (1) hide show
  1. config.json +2 -1
config.json CHANGED
@@ -22,5 +22,6 @@
22
  "torch_dtype": "float16",
23
  "transformers_version": "4.33.0.dev0",
24
  "use_cache": false,
25
- "vocab_size": 32000
 
26
  }
 
22
  "torch_dtype": "float16",
23
  "transformers_version": "4.33.0.dev0",
24
  "use_cache": false,
25
+ "vocab_size": 32000,
26
+ "pad_token_id": 0
27
  }