Hugging Face's logo

Shankhdhar
/
classifier_woog

Text Classification
setfit
Safetensors
sentence-transformers
mpnet
generated_from_setfit_trainer
Eval Results
Model card Files Community
Shankhdhar commited on
Commit
a5b9b7a
·
verified ·
1 Parent(s): ec71f47

Add SetFit model

Browse files
Files changed (4) hide show
  1. README.md +39 -86
  2. config_setfit.json +0 -2
  3. model.safetensors +1 -1
  4. model_head.pkl +2 -2
README.md CHANGED
@@ -9,15 +9,12 @@ base_model: sentence-transformers/paraphrase-mpnet-base-v2
9
  metrics:
10
  - accuracy
11
  widget:
12
- - text: I recently ordered the Bella Silver Pendant, but I haven't received any update
13
- about the shipment. Can you provide me with the current status of my order?
14
- - text: What is the metal purity of the Eternal Swirl Rose Gold Hoop Earring, and
15
- does it come with a certificate of authenticity?
16
- - text: Can you suggest some minimalist necklaces from your 'Best Sellers - Minimalist'
17
- range?
18
- - text: I recently ordered the Pearly Round Earring but haven't received any shipping
19
- updates. Can you please provide me with the tracking information?
20
- - text: what are the colors available in air jordan 4
21
  pipeline_tag: text-classification
22
  inference: true
23
  model-index:
@@ -32,7 +29,7 @@ model-index:
32
  split: test
33
  metrics:
34
  - type: accuracy
35
- value: 0.872852233676976
36
  name: Accuracy
37
  ---
38
 
@@ -52,7 +49,7 @@ The model has been trained using an efficient few-shot learning technique that i
52
  - **Sentence Transformer body:** [sentence-transformers/paraphrase-mpnet-base-v2](https://huggingface.co/sentence-transformers/paraphrase-mpnet-base-v2)
53
  - **Classification head:** a [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance
54
  - **Maximum Sequence Length:** 512 tokens
55
- - **Number of Classes:** 6 classes
56
  <!-- - **Training Dataset:** [Unknown](https://huggingface.co/datasets/unknown) -->
57
  <!-- - **Language:** Unknown -->
58
  <!-- - **License:** Unknown -->
@@ -64,21 +61,19 @@ The model has been trained using an efficient few-shot learning technique that i
64
  - **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit)
65
 
66
  ### Model Labels
67
- | Label | Examples |
68
- |:------------------------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
69
- | product policy | <ul><li>'Are there any exceptions to the return policy for items that were purchased with a special offer promotion?'</li><li>'What is your policy on returning sneakers with added paint or dye?'</li><li>'Do you offer exchanges for items that were purchased with a special event celebration?'</li></ul> |
70
- | order tracking | <ul><li>"I recently placed an order for the Regalia Gold Ring but I haven't received any confirmation or tracking details. Could you please update me on the status of my order?"</li><li>'What is the process for rerouting a shipment to a different address?'</li><li>"I recently ordered a Three Crystal Proposal Ring but haven't received any shipping updates yet. Could you please provide me with the current status of my order?"</li></ul> |
71
- | complaints | <ul><li>"I recently bought the Golden Love Affair Pendant, but it seems to have tarnished very quickly. I'm not satisfied with the quality. What can you do about this?"</li><li>"I recently purchased the Three Crystal Proposal Ring, but I'm disappointed to find that one of the crystals is loose. Can you assist me with this issue?"</li><li>'I received my Kali- Handcrafted Earring today, but I found that one earring is slightly different from the other in design. Can you help me with this issue?'</li></ul> |
72
- | product faq | <ul><li>'What is the material used for making the All the Stars Pendant Set, and does it come with matching earrings?'</li><li>'What is the Bold and Beautiful Link Ring made of, and could you provide information on sizing and care instructions?'</li><li>'What is the material used for making the Sheer Heart Ring, and is it available in different sizes?'</li></ul> |
73
- | product discoveribility | <ul><li>"I'm interested in necklaces that have an adjustable length. What options do you have?"</li><li>'Do you have any charm bracelets available at your store?'</li><li>'Could you suggest some pendants that would go well with traditional attire?'</li></ul> |
74
- | product discoverability | <ul><li>'Types of bakery boxes available'</li><li>'adidas sneakers under 25k'</li><li>'show me 100 cookie boxes under $50'</li></ul> |
75
 
76
  ## Evaluation
77
 
78
  ### Metrics
79
  | Label | Accuracy |
80
  |:--------|:---------|
81
- | **all** | 0.8729 |
82
 
83
  ## Uses
84
 
@@ -98,7 +93,7 @@ from setfit import SetFitModel
98
  # Download from the 🤗 Hub
99
  model = SetFitModel.from_pretrained("setfit_model_id")
100
  # Run inference
101
- preds = model("what are the colors available in air jordan 4")
102
  ```
103
 
104
  <!--
@@ -130,15 +125,13 @@ preds = model("what are the colors available in air jordan 4")
130
  ### Training Set Metrics
131
  | Training set | Min | Median | Max |
132
  |:-------------|:----|:--------|:----|
133
- | Word count | 4 | 16.2235 | 36 |
134
 
135
  | Label | Training Sample Count |
136
  |:------------------------|:----------------------|
137
- | complaints | 30 |
138
  | order tracking | 30 |
139
  | product discoverability | 30 |
140
- | product discoveribility | 30 |
141
- | product faq | 20 |
142
  | product policy | 30 |
143
 
144
  ### Training Hyperparameters
@@ -161,67 +154,27 @@ preds = model("what are the colors available in air jordan 4")
161
  ### Training Results
162
  | Epoch | Step | Training Loss | Validation Loss |
163
  |:------:|:----:|:-------------:|:---------------:|
164
- | 0.0007 | 1 | 0.1501 | - |
165
- | 0.0333 | 50 | 0.1476 | - |
166
- | 0.0667 | 100 | 0.0248 | - |
167
- | 0.1 | 150 | 0.009 | - |
168
- | 0.1333 | 200 | 0.0022 | - |
169
- | 0.1667 | 250 | 0.001 | - |
170
- | 0.2 | 300 | 0.0009 | - |
171
- | 0.2333 | 350 | 0.0007 | - |
172
- | 0.2667 | 400 | 0.0004 | - |
173
- | 0.3 | 450 | 0.0006 | - |
174
- | 0.3333 | 500 | 0.0003 | - |
175
- | 0.3667 | 550 | 0.0003 | - |
176
- | 0.4 | 600 | 0.0002 | - |
177
- | 0.4333 | 650 | 0.0002 | - |
178
- | 0.4667 | 700 | 0.0003 | - |
179
- | 0.5 | 750 | 0.0002 | - |
180
- | 0.5333 | 800 | 0.0003 | - |
181
- | 0.5667 | 850 | 0.0002 | - |
182
- | 0.6 | 900 | 0.0002 | - |
183
- | 0.6333 | 950 | 0.0001 | - |
184
- | 0.6667 | 1000 | 0.0001 | - |
185
- | 0.7 | 1050 | 0.0001 | - |
186
- | 0.7333 | 1100 | 0.0002 | - |
187
- | 0.7667 | 1150 | 0.0002 | - |
188
- | 0.8 | 1200 | 0.0001 | - |
189
- | 0.8333 | 1250 | 0.0001 | - |
190
- | 0.8667 | 1300 | 0.0001 | - |
191
- | 0.9 | 1350 | 0.0001 | - |
192
- | 0.9333 | 1400 | 0.0002 | - |
193
- | 0.9667 | 1450 | 0.0001 | - |
194
- | 1.0 | 1500 | 0.0002 | - |
195
- | 1.0333 | 1550 | 0.0001 | - |
196
- | 1.0667 | 1600 | 0.0001 | - |
197
- | 1.1 | 1650 | 0.0001 | - |
198
- | 1.1333 | 1700 | 0.0001 | - |
199
- | 1.1667 | 1750 | 0.0001 | - |
200
- | 1.2 | 1800 | 0.0001 | - |
201
- | 1.2333 | 1850 | 0.0001 | - |
202
- | 1.2667 | 1900 | 0.0001 | - |
203
- | 1.3 | 1950 | 0.0001 | - |
204
- | 1.3333 | 2000 | 0.0001 | - |
205
- | 1.3667 | 2050 | 0.0001 | - |
206
- | 1.4 | 2100 | 0.0001 | - |
207
- | 1.4333 | 2150 | 0.0001 | - |
208
- | 1.4667 | 2200 | 0.0001 | - |
209
- | 1.5 | 2250 | 0.0001 | - |
210
- | 1.5333 | 2300 | 0.0001 | - |
211
- | 1.5667 | 2350 | 0.0001 | - |
212
- | 1.6 | 2400 | 0.0001 | - |
213
- | 1.6333 | 2450 | 0.0001 | - |
214
- | 1.6667 | 2500 | 0.0001 | - |
215
- | 1.7 | 2550 | 0.0001 | - |
216
- | 1.7333 | 2600 | 0.0001 | - |
217
- | 1.7667 | 2650 | 0.0001 | - |
218
- | 1.8 | 2700 | 0.0001 | - |
219
- | 1.8333 | 2750 | 0.0001 | - |
220
- | 1.8667 | 2800 | 0.0001 | - |
221
- | 1.9 | 2850 | 0.0001 | - |
222
- | 1.9333 | 2900 | 0.0001 | - |
223
- | 1.9667 | 2950 | 0.0001 | - |
224
- | 2.0 | 3000 | 0.0001 | - |
225
 
226
  ### Framework Versions
227
  - Python: 3.9.16
 
9
  metrics:
10
  - accuracy
11
  widget:
12
+ - text: cookie boxes for gifting under $20
13
+ - text: Are there any restrictions on returning candle supplies?
14
+ - text: special features for bakery boxes
15
+ - text: I need to confirm the shipping date for my recent purchase. Can you help me
16
+ with that?
17
+ - text: different types of bakery boxes available
 
 
 
18
  pipeline_tag: text-classification
19
  inference: true
20
  model-index:
 
29
  split: test
30
  metrics:
31
  - type: accuracy
32
+ value: 0.8380952380952381
33
  name: Accuracy
34
  ---
35
 
 
49
  - **Sentence Transformer body:** [sentence-transformers/paraphrase-mpnet-base-v2](https://huggingface.co/sentence-transformers/paraphrase-mpnet-base-v2)
50
  - **Classification head:** a [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance
51
  - **Maximum Sequence Length:** 512 tokens
52
+ - **Number of Classes:** 4 classes
53
  <!-- - **Training Dataset:** [Unknown](https://huggingface.co/datasets/unknown) -->
54
  <!-- - **Language:** Unknown -->
55
  <!-- - **License:** Unknown -->
 
61
  - **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit)
62
 
63
  ### Model Labels
64
+ | Label | Examples |
65
+ |:------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
66
+ | product discoverability | <ul><li>'Do you have Adidas Superstar shoes?'</li><li>'Do you have any running shoes in pink color?'</li><li>'Do you have black Yeezy sneakers in size 9?'</li></ul> |
67
+ | order tracking | <ul><li>"I'm concerned about the delay in the delivery of my order. Can you please provide me with the status?"</li><li>'What is the estimated delivery time for orders within the same city?'</li><li>"I placed an order last week and it still hasn't arrived. Can you check the status for me?"</li></ul> |
68
+ | product policy | <ul><li>'Are there any exceptions to the return policy for items that were purchased with a student discount?'</li><li>'Do you offer a try-and-buy option for sneakers?'</li><li>'Do you offer a price adjustment for sneakers if the price drops after purchase?'</li></ul> |
69
+ | product faq | <ul><li>'Do you have any limited edition sneakers available?'</li><li>'Are the Adidas Yeezy Foam Runner available in size 7?'</li><li>"Are the Nike Air Force 1 sneakers available in women's sizes?"</li></ul> |
 
 
70
 
71
  ## Evaluation
72
 
73
  ### Metrics
74
  | Label | Accuracy |
75
  |:--------|:---------|
76
+ | **all** | 0.8381 |
77
 
78
  ## Uses
79
 
 
93
  # Download from the 🤗 Hub
94
  model = SetFitModel.from_pretrained("setfit_model_id")
95
  # Run inference
96
+ preds = model("special features for bakery boxes")
97
  ```
98
 
99
  <!--
 
125
  ### Training Set Metrics
126
  | Training set | Min | Median | Max |
127
  |:-------------|:----|:--------|:----|
128
+ | Word count | 3 | 11.6415 | 24 |
129
 
130
  | Label | Training Sample Count |
131
  |:------------------------|:----------------------|
 
132
  | order tracking | 30 |
133
  | product discoverability | 30 |
134
+ | product faq | 16 |
 
135
  | product policy | 30 |
136
 
137
  ### Training Hyperparameters
 
154
  ### Training Results
155
  | Epoch | Step | Training Loss | Validation Loss |
156
  |:------:|:----:|:-------------:|:---------------:|
157
+ | 0.0019 | 1 | 0.1782 | - |
158
+ | 0.0965 | 50 | 0.0628 | - |
159
+ | 0.1931 | 100 | 0.0036 | - |
160
+ | 0.2896 | 150 | 0.0013 | - |
161
+ | 0.3861 | 200 | 0.0012 | - |
162
+ | 0.4826 | 250 | 0.0003 | - |
163
+ | 0.5792 | 300 | 0.0002 | - |
164
+ | 0.6757 | 350 | 0.0003 | - |
165
+ | 0.7722 | 400 | 0.0002 | - |
166
+ | 0.8687 | 450 | 0.0005 | - |
167
+ | 0.9653 | 500 | 0.0003 | - |
168
+ | 1.0618 | 550 | 0.0001 | - |
169
+ | 1.1583 | 600 | 0.0002 | - |
170
+ | 1.2548 | 650 | 0.0002 | - |
171
+ | 1.3514 | 700 | 0.0002 | - |
172
+ | 1.4479 | 750 | 0.0001 | - |
173
+ | 1.5444 | 800 | 0.0001 | - |
174
+ | 1.6409 | 850 | 0.0001 | - |
175
+ | 1.7375 | 900 | 0.0002 | - |
176
+ | 1.8340 | 950 | 0.0001 | - |
177
+ | 1.9305 | 1000 | 0.0001 | - |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
178
 
179
  ### Framework Versions
180
  - Python: 3.9.16
config_setfit.json CHANGED
@@ -1,10 +1,8 @@
1
  {
2
  "normalize_embeddings": false,
3
  "labels": [
4
- "complaints",
5
  "order tracking",
6
  "product discoverability",
7
- "product discoveribility",
8
  "product faq",
9
  "product policy"
10
  ]
 
1
  {
2
  "normalize_embeddings": false,
3
  "labels": [
 
4
  "order tracking",
5
  "product discoverability",
 
6
  "product faq",
7
  "product policy"
8
  ]
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:233fa17e41eedd3308c4817e2a79905d71f3a8039592ff44b6b078f597f18d09
3
  size 437967672
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6f4712a705cba776438959c8d112f3aa0a6e1f73cd50b9c18670258c3a70c6c1
3
  size 437967672
model_head.pkl CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:6f19361422bef33cfeb630852d67b9f643c1ce0ef4b71982d521e05502e7fcba
3
- size 38311
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:40c21660c91e8d850fe7d90de2f1bed643f254f1a7c9ff8690e9c637cd7c1e2a
3
+ size 25815