pankajroark
/

llama-fp16-engine

Model card Files Files and versions Community

llama-fp16-engine / 7b-sq-int8kv-tp1

1 contributor

History: 2 commits

pankajroark's picture

inflight batching engine for 7b-sq-int8kv-tp1

2319002 over 1 year ago

config.json

1.31 kB

inflight batching engine for 7b-sq-int8kv-tp1 over 1 year ago
llama_float16_tp1_rank0.engine

7.01 GB
LFS

inflight batching engine for 7b-sq-int8kv-tp1 over 1 year ago
model.cache

96.1 kB

inflight batching engine for 7b-sq-int8kv-tp1 over 1 year ago