Divyasreepat commited on
Commit
d2b0f42
·
verified ·
1 Parent(s): afba855

Update README.md with new model card content

Browse files
Files changed (1) hide show
  1. README.md +143 -14
README.md CHANGED
@@ -1,17 +1,146 @@
1
  ---
2
  library_name: keras-hub
3
  ---
4
- This is a [`DebertaV3` model](https://keras.io/api/keras_hub/models/deberta_v3) uploaded using the KerasHub library and can be used with JAX, TensorFlow, and PyTorch backends.
5
- Model config:
6
- * **name:** deberta_v3_backbone
7
- * **trainable:** True
8
- * **vocabulary_size:** 128100
9
- * **num_layers:** 6
10
- * **num_heads:** 12
11
- * **hidden_dim:** 768
12
- * **intermediate_dim:** 3072
13
- * **dropout:** 0.1
14
- * **max_sequence_length:** 512
15
- * **bucket_size:** 256
16
-
17
- This model card has been generated automatically and should be completed by the model author. See [Model Cards documentation](https://huggingface.co/docs/hub/model-cards) for more information.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  library_name: keras-hub
3
  ---
4
+ ### Model Overview
5
+ DeBERTaV3 encoder networks are a set of transformer encoder models published by Microsoft. DeBERTa improves the BERT and RoBERTa models using disentangled attention and enhanced mask decoder.
6
+
7
+ Weights are released under the [MIT License](https://opensource.org/license/mit). Keras model code is released under the [Apache 2 License](https://github.com/keras-team/keras-hub/blob/master/LICENSE).
8
+
9
+ ## Links
10
+
11
+ * [DeBERTaV3 Quickstart Notebook](https://www.kaggle.com/code/gabrielrasskin/debertav3-quickstart)
12
+ * [DeBERTaV3 API Documentation](https://keras.io/api/keras_hub/models/deberta_v3/deberta_v3_classifier/)
13
+ * [DeBERTaV3 Model Paper](https://arxiv.org/abs/2111.09543)
14
+ * [KerasHub Beginner Guide](https://keras.io/guides/keras_hub/getting_started/)
15
+ * [KerasHub Model Publishing Guide](https://keras.io/guides/keras_hub/upload/)
16
+
17
+ ## Installation
18
+
19
+ Keras and KerasHub can be installed with:
20
+
21
+ ```
22
+ pip install -U -q keras-hub
23
+ pip install -U -q keras>=3
24
+ ```
25
+
26
+ Jax, TensorFlow, and Torch come preinstalled in Kaggle Notebooks. For instruction on installing them in another environment see the [Keras Getting Started](https://keras.io/getting_started/) page.
27
+
28
+ ## Presets
29
+
30
+ The following model checkpoints are provided by the Keras team. Full code examples for each are available below.
31
+
32
+ | Preset Name | Parameters | Description |
33
+ | :------------------------------- | :------------: | :-------------------------------------------------------------------------------------------------------- |
34
+ | `deberta_v3_extra_small_en` | 70.68M | 12-layer DeBERTaV3 model where case is maintained. Trained on English Wikipedia, BookCorpus and OpenWebText. |
35
+ | `deberta_v3_small_en` | 141.30M | 6-layer DeBERTaV3 model where case is maintained. Trained on English Wikipedia, BookCorpus and OpenWebText. |
36
+ | `deberta_v3_base_en` | 183.83M | 12-layer DeBERTaV3 model where case is maintained. Trained on English Wikipedia, BookCorpus and OpenWebText. |
37
+ | `deberta_v3_large_en` | 434.01M | 24-layer DeBERTaV3 model where case is maintained. Trained on English Wikipedia, BookCorpus and OpenWebText. |
38
+ | `deberta_v3_base_multi` | 278.22M | 12-layer DeBERTaV3 model where case is maintained. Trained on the 2.5TB multilingual CC100 dataset. |
39
+
40
+ ## Prompts
41
+
42
+ DeBERTa's main use as a classifier takes in raw text that is labelled by the class it belongs to. In practice this can look like this:
43
+
44
+ ```python
45
+ features = ["The quick brown fox jumped.", "I forgot my homework."]
46
+ labels = [0, 3]
47
+ ```
48
+
49
+ ### Example Usage
50
+ ```python
51
+ import keras
52
+ import keras_hub
53
+ import numpy as np
54
+ ```
55
+
56
+ Raw string data.
57
+ ```python
58
+ features = ["The quick brown fox jumped.", "I forgot my homework."]
59
+ labels = [0, 3]
60
+
61
+ # Pretrained classifier.
62
+ classifier = keras_hub.models.DebertaV3Classifier.from_preset(
63
+ "deberta_v3_small_en",
64
+ num_classes=4,
65
+ )
66
+ classifier.fit(x=features, y=labels, batch_size=2)
67
+ classifier.predict(x=features, batch_size=2)
68
+
69
+ # Re-compile (e.g., with a new learning rate).
70
+ classifier.compile(
71
+ loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),
72
+ optimizer=keras.optimizers.Adam(5e-5),
73
+ jit_compile=True,
74
+ )
75
+ # Access backbone programmatically (e.g., to change `trainable`).
76
+ classifier.backbone.trainable = False
77
+ # Fit again.
78
+ classifier.fit(x=features, y=labels, batch_size=2)
79
+ ```
80
+
81
+ Preprocessed integer data.
82
+ ```python
83
+ features = {
84
+ "token_ids": np.ones(shape=(2, 12), dtype="int32"),
85
+ "padding_mask": np.array([[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0]] * 2),
86
+ }
87
+ labels = [0, 3]
88
+
89
+ # Pretrained classifier without preprocessing.
90
+ classifier = keras_hub.models.DebertaV3Classifier.from_preset(
91
+ "deberta_v3_small_en",
92
+ num_classes=4,
93
+ preprocessor=None,
94
+ )
95
+ classifier.fit(x=features, y=labels, batch_size=2)
96
+ ```
97
+
98
+ ## Example Usage with Hugging Face URI
99
+
100
+ ```python
101
+ import keras
102
+ import keras_hub
103
+ import numpy as np
104
+ ```
105
+
106
+ Raw string data.
107
+ ```python
108
+ features = ["The quick brown fox jumped.", "I forgot my homework."]
109
+ labels = [0, 3]
110
+
111
+ # Pretrained classifier.
112
+ classifier = keras_hub.models.DebertaV3Classifier.from_preset(
113
+ "hf://keras/deberta_v3_small_en",
114
+ num_classes=4,
115
+ )
116
+ classifier.fit(x=features, y=labels, batch_size=2)
117
+ classifier.predict(x=features, batch_size=2)
118
+
119
+ # Re-compile (e.g., with a new learning rate).
120
+ classifier.compile(
121
+ loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),
122
+ optimizer=keras.optimizers.Adam(5e-5),
123
+ jit_compile=True,
124
+ )
125
+ # Access backbone programmatically (e.g., to change `trainable`).
126
+ classifier.backbone.trainable = False
127
+ # Fit again.
128
+ classifier.fit(x=features, y=labels, batch_size=2)
129
+ ```
130
+
131
+ Preprocessed integer data.
132
+ ```python
133
+ features = {
134
+ "token_ids": np.ones(shape=(2, 12), dtype="int32"),
135
+ "padding_mask": np.array([[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0]] * 2),
136
+ }
137
+ labels = [0, 3]
138
+
139
+ # Pretrained classifier without preprocessing.
140
+ classifier = keras_hub.models.DebertaV3Classifier.from_preset(
141
+ "hf://keras/deberta_v3_small_en",
142
+ num_classes=4,
143
+ preprocessor=None,
144
+ )
145
+ classifier.fit(x=features, y=labels, batch_size=2)
146
+ ```