xriminact
/

llama-3-8b-instruct-openvino-int4

Text Generation

Inference Endpoints

Model card Files Files and versions Community

xriminact commited on Apr 23, 2024

Commit

bca33dd

·

verified ·

1 Parent(s): 3ca46c5

Update README.md

Files changed (1) hide show

README.md +1 -2

README.md CHANGED Viewed

@@ -22,9 +22,8 @@ ov_model = OVModelForCausalLM.from_pretrained(
     trust_remote_code=True,
 )
-tokenizer_kwargs = model_configuration.get("tokenizer_kwargs", {})
 test_string = "What is OpenVino?"
-input_tokens = tok(test_string, return_tensors="pt", **tokenizer_kwargs)
 answer = ov_model.generate(**input_tokens, max_new_tokens=200)
 print(tok.batch_decode(answer, skip_special_tokens=True)[0])
 ```

     trust_remote_code=True,
 )
 test_string = "What is OpenVino?"
+input_tokens = tok(test_string, return_tensors="pt")
 answer = ov_model.generate(**input_tokens, max_new_tokens=200)
 print(tok.batch_decode(answer, skip_special_tokens=True)[0])
 ```