ferran-espuna
commited on
Commit
•
70e5ace
1
Parent(s):
f1c1dc8
Update README.md
Browse files
README.md
CHANGED
@@ -61,8 +61,25 @@ This model card corresponds to the gptq-quantized version of Salamandra-2b for s
|
|
61 |
|
62 |
The entire Salamandra family is released under a permissive [Apache 2.0 license]((https://www.apache.org/licenses/LICENSE-2.0)).
|
63 |
|
|
|
64 |
|
65 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
66 |
|
67 |
### Author
|
68 |
International Business Machines (IBM).
|
|
|
61 |
|
62 |
The entire Salamandra family is released under a permissive [Apache 2.0 license]((https://www.apache.org/licenses/LICENSE-2.0)).
|
63 |
|
64 |
+
## How to Use
|
65 |
|
66 |
+
The following example code works under ``Python 3.9.16``, ``vllm==0.6.3.post1``, ``torch==2.4.0`` and ``torchvision==0.19.0``, though it should run on
|
67 |
+
any current version of the libraries. This is an example of how to create a text completion using the model:
|
68 |
+
|
69 |
+
```
|
70 |
+
from vllm import LLM, SamplingParams
|
71 |
+
|
72 |
+
model_name = "BSC-LT/salamandra-2b-base-fp8"
|
73 |
+
llm = LLM(model=model_name)
|
74 |
+
|
75 |
+
outputs = llm.generate("El mercat del barri ",
|
76 |
+
sampling_params=SamplingParams(
|
77 |
+
temperature=0.5,
|
78 |
+
max_tokens=200)
|
79 |
+
)
|
80 |
+
print(outputs[0].outputs[0].text)
|
81 |
+
|
82 |
+
```
|
83 |
|
84 |
### Author
|
85 |
International Business Machines (IBM).
|