CharlieFRuan commited on
Commit
5fdef0d
·
verified ·
1 Parent(s): b80d96f

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +0 -41
README.md CHANGED
@@ -11,47 +11,6 @@ tags:
11
  This is the [snowflake-arctic-embed-s](https://huggingface.co/Snowflake/snowflake-arctic-embed-s) model in MLC format `q0f32`.
12
  The model can be used for projects [MLC-LLM](https://github.com/mlc-ai/mlc-llm) and [WebLLM](https://github.com/mlc-ai/web-llm).
13
 
14
- ## Example Usage
15
-
16
- Here are some examples of using this model in MLC LLM.
17
- Before running the examples, please install MLC LLM by following the [installation documentation](https://llm.mlc.ai/docs/install/mlc_llm.html#install-mlc-packages).
18
-
19
- ### Chat
20
-
21
- In command line, run
22
- ```bash
23
- mlc_llm chat HF://mlc-ai/snowflake-arctic-embed-s-q0f32-MLC
24
- ```
25
-
26
- ### REST Server
27
-
28
- In command line, run
29
- ```bash
30
- mlc_llm serve HF://mlc-ai/snowflake-arctic-embed-s-q0f32-MLC
31
- ```
32
-
33
- ### Python API
34
-
35
- ```python
36
- from mlc_llm import MLCEngine
37
-
38
- # Create engine
39
- model = "HF://mlc-ai/snowflake-arctic-embed-s-q0f32-MLC"
40
- engine = MLCEngine(model)
41
-
42
- # Run chat completion in OpenAI API.
43
- for response in engine.chat.completions.create(
44
- messages=[{"role": "user", "content": "What is the meaning of life?"}],
45
- model=model,
46
- stream=True,
47
- ):
48
- for choice in response.choices:
49
- print(choice.delta.content, end="", flush=True)
50
- print("\n")
51
-
52
- engine.terminate()
53
- ```
54
-
55
  ## Documentation
56
 
57
  For more information on MLC LLM project, please visit our [documentation](https://llm.mlc.ai/docs/) and [GitHub repo](http://github.com/mlc-ai/mlc-llm).
 
11
  This is the [snowflake-arctic-embed-s](https://huggingface.co/Snowflake/snowflake-arctic-embed-s) model in MLC format `q0f32`.
12
  The model can be used for projects [MLC-LLM](https://github.com/mlc-ai/mlc-llm) and [WebLLM](https://github.com/mlc-ai/web-llm).
13
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
14
  ## Documentation
15
 
16
  For more information on MLC LLM project, please visit our [documentation](https://llm.mlc.ai/docs/) and [GitHub repo](http://github.com/mlc-ai/mlc-llm).