Update README.md
Browse files
README.md
CHANGED
|
@@ -8,6 +8,8 @@ tags: []
|
|
| 8 |
|
| 9 |
Requires PyTorch 2.5+ for the most savings, see [SDPA](https://pytorch.org/docs/stable/generated/torch.nn.functional.scaled_dot_product_attention.html).
|
| 10 |
|
|
|
|
|
|
|
| 11 |
```python
|
| 12 |
import torch
|
| 13 |
from transformers import AutoModel, AutoTokenizer
|
|
|
|
| 8 |
|
| 9 |
Requires PyTorch 2.5+ for the most savings, see [SDPA](https://pytorch.org/docs/stable/generated/torch.nn.functional.scaled_dot_product_attention.html).
|
| 10 |
|
| 11 |
+
Outputting attentions and predicting contacts are not possible from SDPA. Various other optimizations also make the base implementation slightly different than the HF one.
|
| 12 |
+
|
| 13 |
```python
|
| 14 |
import torch
|
| 15 |
from transformers import AutoModel, AutoTokenizer
|