Divyasreepat commited on
Commit
44ea95f
1 Parent(s): fb39ad6

Update README.md with new model card content

Browse files
Files changed (1) hide show
  1. README.md +131 -13
README.md CHANGED
@@ -1,16 +1,134 @@
1
  ---
2
  library_name: keras-hub
3
  ---
4
- This is a [`FNet` model](https://keras.io/api/keras_hub/models/f_net) uploaded using the KerasHub library and can be used with JAX, TensorFlow, and PyTorch backends.
5
- Model config:
6
- * **name:** f_net_backbone
7
- * **trainable:** True
8
- * **vocabulary_size:** 32000
9
- * **num_layers:** 24
10
- * **hidden_dim:** 1024
11
- * **intermediate_dim:** 4096
12
- * **dropout:** 0.1
13
- * **max_sequence_length:** 512
14
- * **num_segments:** 4
15
-
16
- This model card has been generated automatically and should be completed by the model author. See [Model Cards documentation](https://huggingface.co/docs/hub/model-cards) for more information.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  library_name: keras-hub
3
  ---
4
+ ### Model Overview
5
+ FNet is a set of language models published by Google as part of the paper [FNet: Mixing Tokens with Fourier Transforms](https://arxiv.org/abs/2105.03824). FNet replaces the self-attention of BERT with an unparameterized fourier transform, dramatically lowering the number of trainable parameters in the model. FNet achieves training at 92-97% accuracy of BERT counterparts on GLUE benchmark, with faster training and much smaller saved checkpoints.
6
+
7
+ Weights and Keras model code are released under the [Apache 2 License](https://github.com/keras-team/keras-hub/blob/master/LICENSE).
8
+
9
+ ## Links
10
+
11
+ * [FNet Quickstart Notebook](https://www.kaggle.com/code/matthewdwatson/fnet-quickstart/)
12
+ * [FNet API Documentation](https://keras.io/api/keras_hub/models/f_net/)
13
+ * [FNet Model Card](https://github.com/google-research/google-research/blob/master/f_net/README.md)
14
+ * [KerasHub Beginner Guide](https://keras.io/guides/keras_hub/getting_started/)
15
+ * [KerasHub Model Publishing Guide](https://keras.io/guides/keras_hub/upload/)
16
+
17
+ ## Installation
18
+
19
+ Keras and KerasHub can be installed with:
20
+
21
+ ```
22
+ pip install -U -q keras-hub
23
+ pip install -U -q keras>=3
24
+ ```
25
+
26
+ Jax, TensorFlow, and Torch come preinstalled in Kaggle Notebooks. For instruction on installing them in another environment see the [Keras Getting Started](https://keras.io/getting_started/) page.
27
+
28
+ ## Presets
29
+
30
+ The following model checkpoints are provided by the Keras team. Full code examples for each are available below.
31
+
32
+ | Preset name | Parameters | Description |
33
+ |----------------|------------|-----------------------------------------------|
34
+ | `f_net_base_en` | 82.86M | 12-layer FNet model where case is maintained. |
35
+ | `f_net_large_en` | 236.95M | 24-layer FNet model where case is maintained. |
36
+
37
+ ### Example Usage
38
+ ```python
39
+ import keras
40
+ import keras_hub
41
+ import numpy as np
42
+ ```
43
+
44
+ Raw string data.
45
+ ```python
46
+ features = ["The quick brown fox jumped.", "I forgot my homework."]
47
+ labels = [0, 3]
48
+
49
+ # Pretrained classifier.
50
+ classifier = keras_hub.models.FNetClassifier.from_preset(
51
+ "f_net_base_en",
52
+ num_classes=4,
53
+ )
54
+ classifier.fit(x=features, y=labels, batch_size=2)
55
+ classifier.predict(x=features, batch_size=2)
56
+
57
+ # Re-compile (e.g., with a new learning rate).
58
+ classifier.compile(
59
+ loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),
60
+ optimizer=keras.optimizers.Adam(5e-5),
61
+ jit_compile=True,
62
+ )
63
+ # Access backbone programmatically (e.g., to change `trainable`).
64
+ classifier.backbone.trainable = False
65
+ # Fit again.
66
+ classifier.fit(x=features, y=labels, batch_size=2)
67
+ ```
68
+
69
+ Preprocessed integer data.
70
+ ```python
71
+ features = {
72
+ "token_ids": np.ones(shape=(2, 12), dtype="int32"),
73
+ "segment_ids": np.array([[0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0]] * 2),
74
+ }
75
+ labels = [0, 3]
76
+
77
+ # Pretrained classifier without preprocessing.
78
+ classifier = keras_hub.models.FNetClassifier.from_preset(
79
+ "f_net_base_en",
80
+ num_classes=4,
81
+ preprocessor=None,
82
+ )
83
+ classifier.fit(x=features, y=labels, batch_size=2)
84
+ ```
85
+
86
+ ## Example Usage with Hugging Face URI
87
+
88
+ ```python
89
+ import keras
90
+ import keras_hub
91
+ import numpy as np
92
+ ```
93
+
94
+ Raw string data.
95
+ ```python
96
+ features = ["The quick brown fox jumped.", "I forgot my homework."]
97
+ labels = [0, 3]
98
+
99
+ # Pretrained classifier.
100
+ classifier = keras_hub.models.FNetClassifier.from_preset(
101
+ "f_net_base_en",
102
+ num_classes=4,
103
+ )
104
+ classifier.fit(x=features, y=labels, batch_size=2)
105
+ classifier.predict(x=features, batch_size=2)
106
+
107
+ # Re-compile (e.g., with a new learning rate).
108
+ classifier.compile(
109
+ loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),
110
+ optimizer=keras.optimizers.Adam(5e-5),
111
+ jit_compile=True,
112
+ )
113
+ # Access backbone programmatically (e.g., to change `trainable`).
114
+ classifier.backbone.trainable = False
115
+ # Fit again.
116
+ classifier.fit(x=features, y=labels, batch_size=2)
117
+ ```
118
+
119
+ Preprocessed integer data.
120
+ ```python
121
+ features = {
122
+ "token_ids": np.ones(shape=(2, 12), dtype="int32"),
123
+ "segment_ids": np.array([[0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0]] * 2),
124
+ }
125
+ labels = [0, 3]
126
+
127
+ # Pretrained classifier without preprocessing.
128
+ classifier = keras_hub.models.FNetClassifier.from_preset(
129
+ "f_net_base_en",
130
+ num_classes=4,
131
+ preprocessor=None,
132
+ )
133
+ classifier.fit(x=features, y=labels, batch_size=2)
134
+ ```