ybelkada
/

chatglm3-6b-hf

Text Generation

Inference Endpoints

Model card Files Files and versions Community

ybelkada commited on Dec 29, 2023

Commit

d2c1632

·

1 Parent(s): 58675f8

Update modeling_chatglm.py

Files changed (1) hide show

modeling_chatglm.py +2 -2

modeling_chatglm.py CHANGED Viewed

@@ -871,7 +871,7 @@ class ChatGLMModel(ChatGLMPreTrainedModel):
                 layer_ret = torch.utils.checkpoint.checkpoint(
                     layer,
                     hidden_states,
-                    attention_mask,
                     rotary_pos_emb,
                     past_key_values[index],
                     use_cache
@@ -879,7 +879,7 @@ class ChatGLMModel(ChatGLMPreTrainedModel):
             else:
                 layer_ret = layer(
                     hidden_states,
-                    attention_mask,
                     rotary_pos_emb,
                     kv_cache=past_key_values[index],
                     use_cache=use_cache

                 layer_ret = torch.utils.checkpoint.checkpoint(
                     layer,
                     hidden_states,
+                    full_attention_mask,
                     rotary_pos_emb,
                     past_key_values[index],
                     use_cache
             else:
                 layer_ret = layer(
                     hidden_states,
+                    full_attention_mask,
                     rotary_pos_emb,
                     kv_cache=past_key_values[index],
                     use_cache=use_cache