moshew commited on
Commit
cd4126b
·
1 Parent(s): 6448dc5

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +46 -5
README.md CHANGED
@@ -2,17 +2,58 @@
2
  library_name: keras
3
  ---
4
 
 
 
5
  ## Model description
6
 
7
- More information needed
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
8
 
9
- ## Intended uses & limitations
 
 
10
 
11
- More information needed
12
 
13
- ## Training and evaluation data
14
 
15
- More information needed
16
 
17
  ## Training procedure
18
 
 
2
  library_name: keras
3
  ---
4
 
5
+ x100 smaller with less than 0.5 accuracy drop vs. distilbert-base-uncased-finetuned-sst-2-english
6
+
7
  ## Model description
8
 
9
+ 2 Layers Bilstm model finetuned on SST-2 and distlled from RoBERTa teacher
10
+
11
+ distilbert-base-uncased-finetuned-sst-2-english: 92.2 accuracy, 67M parameters
12
+ moshew/distilbilstm-finetuned-sst-2-english: 91.9 accuracy, 66K parameters
13
+
14
+ ## How to get started with the model
15
+
16
+ Example on SST-2 test dataset classification:
17
+ ​​
18
+ ```python
19
+ from datasets import load_dataset
20
+ import numpy as np
21
+ from sklearn.metrics import accuracy_score
22
+
23
+ from keras.preprocessing.text import Tokenizer
24
+ from keras.utils import pad_sequences
25
+ import tensorflow as tf
26
+
27
+ from huggingface_hub import from_pretrained_keras
28
+
29
+ from datasets import load_dataset
30
+ sst2 = load_dataset("SetFit/sst2")
31
+ augmented_sst2_dataset = load_dataset("jmamou/augmented-glue-sst2")
32
+
33
+ oov_token = '<UNK>' # Required only if test is not given
34
+ pad_type = 'post'
35
+ trunc_type = 'post'
36
+
37
+ # Tokenize our training data
38
+ tokenizer = Tokenizer(num_words=10000)
39
+ tokenizer.fit_on_texts(augmented_sst2_dataset['train']['sentence'])
40
+
41
+ # Encode training data sentences into sequences
42
+ test_sequences = tokenizer.texts_to_sequences(sst2['test']['text'])
43
+
44
+ # Pad the training sequences
45
+ test_padded = pad_sequences(test_sequences, padding=pad_type, truncating=trunc_type, maxlen=64)
46
+
47
+ reloaded_model = from_pretrained_keras('moshew/distilbilstm-finetuned-sst-2-english')
48
 
49
+ pred=reloaded_model.predict(test_padded)
50
+ pred_bin = np.argmax(pred,1)
51
+ accuracy_score(pred_bin, sst2['test']['label'])
52
 
 
53
 
54
+ ```
55
 
56
+ 0.9187259747391543
57
 
58
  ## Training procedure
59