Models
Datasets
Spaces
Posts
Docs
Enterprise
Pricing
Log In
Sign Up

pankajroark
/

llama-fp16-engine

Model card Files Files and versions Community

llama-fp16-engine

1 contributor

History: 9 commits

pankajroark's picture

update engine with inflight options for 7b-sq-int8kv-tp8

ad987a1 over 1 year ago

7b-no-quant-tp1
update non-quant engine again, this time with one that is locally tested over 1 year ago
7b-sq-int8kv-tp1
inflight batching engine for 7b-sq-int8kv-tp1 over 1 year ago
7b-sq-int8kv-tp8
update engine with inflight options for 7b-sq-int8kv-tp8 over 1 year ago
.gitattributes

1.56 kB

checkpoint over 1 year ago
.gitignore

5 Bytes

checkpoint over 1 year ago