|
--- |
|
license: apache-2.0 |
|
--- |
|
|
|
Download a Model |
|
|
|
``` |
|
cortex models pull tinyllama |
|
``` |
|
|
|
Start the Engine |
|
|
|
``` |
|
cortex engines |
|
``` |
|
|
|
Start model |
|
|
|
``` |
|
cortex models start tinyllama |
|
``` |
|
|
|
|
|
Chat |
|
|
|
``` |
|
cortex chat |
|
``` |
|
|
|
Model Structure |
|
|
|
``` |
|
1.1b-chat-v1.0-gguf-Q2_K |
|
1.1b-chat-v1.0-gguf-Q4_K_M |
|
1.1b-chat-v1.0-gguf-Q6_K |
|
1.1b-chat-v1.0-gguf-Q8_0 |
|
1.1b-chat-v1.0-tensorrtllm-windows-ada-int4 |
|
1.1b-chat-v1.0-tensorrtllm-windows-turing-int4 |
|
1.1b-chat-v1.0-tensorrtllm-windows-ampere-int4 |
|
1.1b-chat-v1.0-tensorrtllm-windows-ada-fp16 |
|
1.1b-chat-v1.0-tensorrtllm-windows-turing-fp16 |
|
1.1b-chat-v1.0-tensorrtllm-windows-ampere-fp16 |
|
``` |
