pankajroark
/

llama-fp16-engine

Model card Files Files and versions Community

llama-fp16-engine / 7b-sq-int8kv-tp8

1 contributor

History: 2 commits

pankajroark's picture

update engine with inflight options for 7b-sq-int8kv-tp8

ad987a1 over 1 year ago

config.json

1.32 kB

update engine with inflight options for 7b-sq-int8kv-tp8 over 1 year ago
llama_float16_tp8_rank0.engine

1.11 GB
LFS

update engine with inflight options for 7b-sq-int8kv-tp8 over 1 year ago
llama_float16_tp8_rank1.engine

1.11 GB
LFS

update engine with inflight options for 7b-sq-int8kv-tp8 over 1 year ago
llama_float16_tp8_rank2.engine

1.11 GB
LFS

update engine with inflight options for 7b-sq-int8kv-tp8 over 1 year ago
llama_float16_tp8_rank3.engine

1.11 GB
LFS

update engine with inflight options for 7b-sq-int8kv-tp8 over 1 year ago
llama_float16_tp8_rank4.engine

1.11 GB
LFS

update engine with inflight options for 7b-sq-int8kv-tp8 over 1 year ago
llama_float16_tp8_rank5.engine

1.11 GB
LFS

update engine with inflight options for 7b-sq-int8kv-tp8 over 1 year ago
llama_float16_tp8_rank6.engine

1.11 GB
LFS

update engine with inflight options for 7b-sq-int8kv-tp8 over 1 year ago
llama_float16_tp8_rank7.engine

1.11 GB
LFS

update engine with inflight options for 7b-sq-int8kv-tp8 over 1 year ago
model.cache

109 kB

update engine with inflight options for 7b-sq-int8kv-tp8 over 1 year ago