doberst commited on
Commit
5d17234
·
verified ·
1 Parent(s): a1ec97f

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +6 -7
README.md CHANGED
@@ -3,24 +3,23 @@ license: apache-2.0
3
  inference: false
4
  ---
5
 
6
- BLING-QWEN-NANO-TOOL
7
 
8
 
9
- **bling-qwen-nano-tool** is a RAG-finetuned version on Qwen2-0.5B for use in fact-based context question-answering, packaged with 4_K_M GGUF quantization, providing a very fast, very small inference implementation for use on CPUs.
10
 
11
  To pull the model via API:
12
 
13
  from huggingface_hub import snapshot_download
14
- snapshot_download("llmware/bling-qwen-nano-tool", local_dir="/path/on/your/machine/", local_dir_use_symlinks=False)
15
 
16
-
17
  Load in your favorite GGUF inference engine, or try with llmware as follows:
18
 
19
  from llmware.models import ModelCatalog
20
- model = ModelCatalog().load_model("bling-qwen-nano-tool")
21
- response = model.inference(query, add_context=text_sample)
22
 
23
- Note: please review [**config.json**](https://huggingface.co/llmware/bling-qwen-nano-tool/blob/main/config.json) in the repository for prompt wrapping information, details on the model, and full test set.
24
 
25
 
26
  ### Model Description
 
3
  inference: false
4
  ---
5
 
6
+ BLING-QWEN-MINI-TOOL (1.5B)
7
 
8
 
9
+ **bling-qwen-mini-tool** is a RAG-finetuned version on Qwen2-1.5B for use in fact-based context question-answering, packaged with 4_K_M GGUF quantization, providing a very fast, very small inference implementation for use on CPUs.
10
 
11
  To pull the model via API:
12
 
13
  from huggingface_hub import snapshot_download
14
+ snapshot_download("llmware/bling-qwen-mini-tool", local_dir="/path/on/your/machine/", local_dir_use_symlinks=False)
15
 
 
16
  Load in your favorite GGUF inference engine, or try with llmware as follows:
17
 
18
  from llmware.models import ModelCatalog
19
+ model = ModelCatalog().load_model("bling-qwen-mini-tool")
20
+ response = model.inference(query, add_context=text_sample)
21
 
22
+ Note: please review [**config.json**](https://huggingface.co/llmware/bling-qwen-mini-tool/blob/main/config.json) in the repository for prompt wrapping information, details on the model, and full test set.
23
 
24
 
25
  ### Model Description