|
--- |
|
license: apache-2.0 |
|
--- |
|
|
|
Download a Model |
|
|
|
``` |
|
cortex models pull mistral |
|
``` |
|
|
|
Start the Engine |
|
|
|
``` |
|
cortex engines |
|
|
|
``` |
|
|
|
|
|
Start model |
|
|
|
``` |
|
cortex models start mistral |
|
``` |
|
|
|
|
|
Chat |
|
|
|
``` |
|
cortex chat |
|
``` |
|
|
|
Model Structure |
|
``` |
|
7b-instruct-v0.1-gguf-Q2_K |
|
7b-instruct-v0.1-gguf-Q4_K_M |
|
7b-instruct-v0.1-gguf-Q6_K |
|
7b-instruct-v0.1-gguf-Q8_0 |
|
7b-instruct-v0.2-gguf-Q2_K |
|
7b-instruct-v0.2-gguf-Q4_K_M |
|
7b-instruct-v0.2-gguf-Q6_K |
|
7b-instruct-v0.2-gguf-Q8_0 |
|
7b-instruct-v0.1-tensorrtllm-windows-ada-int4 |
|
7b-instruct-v0.1-tensorrtllm-windows-turing-int4 |
|
7b-instruct-v0.1-tensorrtllm-windows-ampere-int4 |
|
``` |
