| tags: | |
| - deepsparse | |
| # open_llama_3b_v2-ds | |
| https://huggingface.co/openlm-research/open_llama_3b_v2 | |
| ## Inference | |
| ```python | |
| import deepsparse | |
| model = deepsparse.TextGeneration(model="hf:mgoin/open_llama_3b_v2-ds") | |
| model("Tell me a joke.") | |
| ``` | |
| ### Export | |
| ``` | |
| git clone https://huggingface.co/openlm-research/open_llama_3b_v2 | |
| sparseml.transformers.export_onnx --model_path ./open_llama_3b_v2 --task text-generation --sequence_length 512 | |
| ``` |