There is an inconsistency in the model.
#12
by
Kurtulay
- opened
This comment has been hidden
Kurtulay
changed discussion title from
How can inference be done with Python?
to There is an inconsistency in the model.
I can only obtain the desirable results with their mistral-inference lib though.
Using the huggingface version with huggingface transformers lib cannot generate expected results.
Specifically, I received "pass" as the answer for nearly all the problems in HumanEval dataset.