microsoft
/

phi-1_5

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

phi-1_5 / modeling_mixformer_sequential.py

Commit History

Fixes flash-attn import with a try/except statement

0254d42

gugarosa commited on Nov 1, 2023

Adds support for flash-attn rotary embedding and fused dense layers.

0bbd68a

gugarosa commited on Nov 1, 2023

Adds support for MQA/GQA and attention mask during training.

de35f90

gugarosa commited on Oct 30, 2023

Update modeling_mixformer_sequential.py

d38e6f9

gugarosa commited on Oct 26, 2023

Adding _set_gradient_checkpointing for compatibility (#22)

8091327

vriveras commited on Oct 17, 2023

Upload modeling_mixformer_sequential.py

b6a7e2f

gugarosa commited on Sep 27, 2023

fix(phi-1_5): Checks length of `attention_mask`if it is passed as direct tensor.

f9f2ac7

gugarosa commited on Sep 26, 2023

Support for `attention_mask` in forward pass.

3128bb6

gugarosa commited on Sep 26, 2023

add _no_split_modules property (#17)

4a426d8

winglian commited on Sep 15, 2023

Upload MixFormerSequentialForCausalLM

d655135

suriyagunasekar commited on Sep 11, 2023

Upload MixFormerSequentialForCausalLM

1698206

suriyagunasekar commited on Sep 10, 2023