Update README.md
Browse files
README.md
CHANGED
@@ -1,3 +1,48 @@
|
|
1 |
-
---
|
2 |
-
license: apache-2.0
|
3 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
---
|
2 |
+
license: apache-2.0
|
3 |
+
base_model:
|
4 |
+
- HuggingFaceTB/SmolLM3-3B
|
5 |
+
pipeline_tag: text-generation
|
6 |
+
library_name: optimum-executorch
|
7 |
+
tags:
|
8 |
+
- executorch
|
9 |
+
- transformers
|
10 |
+
- optimum-executorch
|
11 |
+
- smollm
|
12 |
+
---
|
13 |
+
|
14 |
+
# Run on-device with ExecuTorch
|
15 |
+
|
16 |
+
This optimized model is exported to ExecuTorch and can run on edge devices.
|
17 |
+
Once ExecuTorch is [set-up](https://pytorch.org/executorch/main/getting-started.html), you can directly download the `*.pte` and tokenizer file and run the model in a mobile app (see [Running in a mobile app](#running-in-a-mobile-app)).
|
18 |
+
|
19 |
+
|
20 |
+
## Export to ExecuTorch
|
21 |
+
|
22 |
+
First need to install the required packages:
|
23 |
+
```Shell
|
24 |
+
pip install git+https://github.com/huggingface/optimum-executorch@main
|
25 |
+
```
|
26 |
+
Then update the dependencies to latest in order to work on the SmolLM3-3B:
|
27 |
+
```Py
|
28 |
+
python install_dev.py
|
29 |
+
```
|
30 |
+
|
31 |
+
Use `optimum-cli` to export the model to ExecuTorch:
|
32 |
+
```Shell
|
33 |
+
optimum-cli export executorch \
|
34 |
+
--model HuggingFaceTB/SmolLM3-3B \
|
35 |
+
--task text-generation \
|
36 |
+
--recipe xnnpack \
|
37 |
+
--use_custom_sdpa \
|
38 |
+
--use_custom_kv_cache \
|
39 |
+
--qlinear \
|
40 |
+
--qembedding \
|
41 |
+
--output_dir ./smollm3_3b
|
42 |
+
```
|
43 |
+
|
44 |
+
|
45 |
+
# Disclaimer
|
46 |
+
PyTorch has not performed safety evaluations or red teamed the quantized models. Performance characteristics, outputs, and behaviors may differ from the original models. Users are solely responsible for selecting appropriate use cases, evaluating and mitigating for accuracy, safety, and fairness, ensuring security, and complying with all applicable laws and regulations.
|
47 |
+
|
48 |
+
Nothing contained in this Model Card should be interpreted as or deemed a restriction or modification to the licenses the models are released under, including any limitations of liability or disclaimers of warranties provided therein.
|