guangy10 commited on
Commit
de19084
·
verified ·
1 Parent(s): 3b8fdc4

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +48 -3
README.md CHANGED
@@ -1,3 +1,48 @@
1
- ---
2
- license: apache-2.0
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ base_model:
4
+ - HuggingFaceTB/SmolLM3-3B
5
+ pipeline_tag: text-generation
6
+ library_name: optimum-executorch
7
+ tags:
8
+ - executorch
9
+ - transformers
10
+ - optimum-executorch
11
+ - smollm
12
+ ---
13
+
14
+ # Run on-device with ExecuTorch
15
+
16
+ This optimized model is exported to ExecuTorch and can run on edge devices.
17
+ Once ExecuTorch is [set-up](https://pytorch.org/executorch/main/getting-started.html), you can directly download the `*.pte` and tokenizer file and run the model in a mobile app (see [Running in a mobile app](#running-in-a-mobile-app)).
18
+
19
+
20
+ ## Export to ExecuTorch
21
+
22
+ First need to install the required packages:
23
+ ```Shell
24
+ pip install git+https://github.com/huggingface/optimum-executorch@main
25
+ ```
26
+ Then update the dependencies to latest in order to work on the SmolLM3-3B:
27
+ ```Py
28
+ python install_dev.py
29
+ ```
30
+
31
+ Use `optimum-cli` to export the model to ExecuTorch:
32
+ ```Shell
33
+ optimum-cli export executorch \
34
+ --model HuggingFaceTB/SmolLM3-3B \
35
+ --task text-generation \
36
+ --recipe xnnpack \
37
+ --use_custom_sdpa \
38
+ --use_custom_kv_cache \
39
+ --qlinear \
40
+ --qembedding \
41
+ --output_dir ./smollm3_3b
42
+ ```
43
+
44
+
45
+ # Disclaimer
46
+ PyTorch has not performed safety evaluations or red teamed the quantized models. Performance characteristics, outputs, and behaviors may differ from the original models. Users are solely responsible for selecting appropriate use cases, evaluating and mitigating for accuracy, safety, and fairness, ensuring security, and complying with all applicable laws and regulations.
47
+
48
+ Nothing contained in this Model Card should be interpreted as or deemed a restriction or modification to the licenses the models are released under, including any limitations of liability or disclaimers of warranties provided therein.