Commit
•
ecf35c4
1
Parent(s):
148ea95
Update README.md
Browse files
README.md
CHANGED
|
@@ -6,6 +6,7 @@ language:
|
|
| 6 |
tags:
|
| 7 |
- text-classification
|
| 8 |
- keras
|
|
|
|
| 9 |
---
|
| 10 |
## Model Overview
|
| 11 |
FNet is a set of language models published by Google as part of the paper [FNet: Mixing Tokens with Fourier Transforms](https://arxiv.org/abs/2105.03824). FNet replaces the self-attention of BERT with an unparameterized fourier transform, dramatically lowering the number of trainable parameters in the model. FNet achieves training at 92-97% accuracy of BERT counterparts on GLUE benchmark, with faster training and much smaller saved checkpoints.
|
|
|
|
| 6 |
tags:
|
| 7 |
- text-classification
|
| 8 |
- keras
|
| 9 |
+
pipeline_tag: text-classification
|
| 10 |
---
|
| 11 |
## Model Overview
|
| 12 |
FNet is a set of language models published by Google as part of the paper [FNet: Mixing Tokens with Fourier Transforms](https://arxiv.org/abs/2105.03824). FNet replaces the self-attention of BERT with an unparameterized fourier transform, dramatically lowering the number of trainable parameters in the model. FNet achieves training at 92-97% accuracy of BERT counterparts on GLUE benchmark, with faster training and much smaller saved checkpoints.
|