Update README.md
Browse files
README.md
CHANGED
@@ -718,10 +718,14 @@ widget:
|
|
718 |
# BEE-spoke-data/bert-plus-L8-4096-v1.0
|
719 |
|
720 |
|
721 |
-
|
|
|
722 |
|
723 |
> still running some evals, etc. expect the model card to change a bit
|
724 |
|
|
|
|
|
|
|
725 |
## this checkpoint
|
726 |
|
727 |
Further progression after multitask training etc. The most recent/last dataset it saw was the euirim/goodwiki dataset.
|
|
|
718 |
# BEE-spoke-data/bert-plus-L8-4096-v1.0
|
719 |
|
720 |
|
721 |
+
|
722 |
+
![image/png](https://cdn-uploads.huggingface.co/production/uploads/60bccec062080d33f875cd0c/I8H0mYfChncerfvtRgLyd.png)
|
723 |
|
724 |
> still running some evals, etc. expect the model card to change a bit
|
725 |
|
726 |
+
\* No additional code. This model uses `attention_type="relative_key"` to help manage the longer ctx.
|
727 |
+
|
728 |
+
|
729 |
## this checkpoint
|
730 |
|
731 |
Further progression after multitask training etc. The most recent/last dataset it saw was the euirim/goodwiki dataset.
|