COCOM_disabled_flash_attn / generation_config.json
Tu2003716's picture
Fine tuned to add key to disable flash_attn for older GPU
027ecf9 verified
raw
history blame
99 Bytes
{
"_from_model_config": true,
"max_new_tokens": 128,
"transformers_version": "4.45.2"
}