---
library_name: setfit
tags:
- setfit
- sentence-transformers
- text-classification
- generated_from_setfit_trainer
metrics:
- accuracy
widget:
- text: okay so just inform them to re submit the order okay because that is the information
that i see right now for the cellphone phone number aah ending again one four
seven one there is the port protection so provide them again the aah aah aah port
out number to port ahead to call this one the transfer pin and they will have
to aah started over and reset make the order and i usually that should really
work
- text: hi welcome to cricket nation thanks for your patience this call may i have
your name
- text: okay okay okay let me just go ahead and wait for the car all right i have
now pulled up of the account and as i can see here the account has uh one line
under the sixty dollar plan and the due date is every the fourth of them i let
me just review the account as well as the payments on the accounts okay can i
just uh put the call on hold for just a minute or two
- text: cleveland but it could be done in the morning or tending that you've name
but it is within today okay so that's a the status for the other number which
is the one ending in one four seven one
- text: ' that yes you did receive two text messages from cricket letting you know
that your payment is due by the nine which i''m i''m not sure how exactly that
was sent to you however prize'
pipeline_tag: text-classification
inference: true
base_model: sentence-transformers/all-mpnet-base-v2
model-index:
- name: SetFit with sentence-transformers/all-mpnet-base-v2
results:
- task:
type: text-classification
name: Text Classification
dataset:
name: Unknown
type: unknown
split: test
metrics:
- type: accuracy
value: 0.8571428571428571
name: Accuracy
---
# SetFit with sentence-transformers/all-mpnet-base-v2
This is a [SetFit](https://github.com/huggingface/setfit) model that can be used for Text Classification. This SetFit model uses [sentence-transformers/all-mpnet-base-v2](https://huggingface.co/sentence-transformers/all-mpnet-base-v2) as the Sentence Transformer embedding model. A [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance is used for classification.
The model has been trained using an efficient few-shot learning technique that involves:
1. Fine-tuning a [Sentence Transformer](https://www.sbert.net) with contrastive learning.
2. Training a classification head with features from the fine-tuned Sentence Transformer.
## Model Details
### Model Description
- **Model Type:** SetFit
- **Sentence Transformer body:** [sentence-transformers/all-mpnet-base-v2](https://huggingface.co/sentence-transformers/all-mpnet-base-v2)
- **Classification head:** a [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance
- **Maximum Sequence Length:** 384 tokens
- **Number of Classes:** 7 classes
### Model Sources
- **Repository:** [SetFit on GitHub](https://github.com/huggingface/setfit)
- **Paper:** [Efficient Few-Shot Learning Without Prompts](https://arxiv.org/abs/2209.11055)
- **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit)
### Model Labels
| Label | Examples |
|:------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| 0 | - "right okay aah i do see here that you got two other numbers two other lines one is ending in zero one eight six and the last one is ending in three two six six i've got here that the bogo is four hundred and thirteen dollars and"
- "gotcha i'm really sure that this is cause a lot of trouble just then let me just pull up your account here and see what's going on so obviously you calling about the number ending in six one two three"
- "okay so i will check the aah status of the aah aah port request okay let me check here okay so for the um the phone number ending in one one zero zero okay i'll check the status for that so it says here that um the new you service provider is all we are waiting for the aah new service provider to activate so uh they will just need to activate that i believe there is a due date of that would be around today okay maybe aah ten in the morning it did not indicate here"
|
| 4 | - "okay so i will check the aah status of the aah aah port request okay let me check here okay so for the um the phone number ending in one one zero zero okay i'll check the status for that so it says here that um the new you service provider is all we are waiting for the aah new service provider to activate so uh they will just need to activate that i believe there is a due date of that would be around today okay maybe aah ten in the morning it did not indicate here"
- "alright got your account who you change that you have uh one line only as per checking here i know that you just paid your bill it's it's because today is your due date"
- "okay aah i got a text aah actually right i actually saw it right before the text it says it my payment or my bill was due okay at midnight at i solve it but and then not on aah on the second or the last my son my phones not working right now as a standard but then i got a text right after that aah on the same aah you know text from cricket and it says that my own read it to you this is what they're me off this is why didn't pay it because aah actually it's kind of a for the aah this is another my phone went out that's another thing i need to ask you about get another phone i i do pay insurance a phone in screen went out anyway on that no it says yeah you okay can i get the payment it says the current balance is zero dollars your next bill is due on twelve nine twenty two sir i thought i had to the ninth i m s is clear they actually take me that's why aah know your and so i didn't think i had to pay and so i didn't think i had to pay it cut me off and now aah that's where i'm at and i was trying to pay it with my card i have on file the service something's wrong i don't know what's goin' on"
|
| 6 | - "did you have the agency apply to your account oh as far as in your bill monthly aah you know your new sixty dollar plan i'm sorry you wanted be on the thirty dollar plan okay so on the sixty dollar and then you only paid thirty on the the bill so thirty five is matching here the remaining balance on the account and the data turned back on on aah would it be possible for us to put her on the thirty dollar rate plan n"
- "oh what do you need to pay okay right now to upgrade to the sixty dollar is gonna be a prorated charge which is let me try to check um hold on oh it's just seventeen cents i can take care of that for you no need to pay today so i'll i'll take care of seventeen cents that is so a prorated amount it will upgrade your plan to sixty dollar and you will have a uh you will get a free feature of fifteen gigabyte hot spot you can enjoy today and then aah i can change it back future dated"
- "all right i'm still updating your plan just give me one moment all right so you see when i upgrade your plan there is going to be a remaining balance of five dollars here so i just need to change it back to fifty five p peter dated at the end of your cycle which is the my so you will not be charged the five dollars"
|
| 5 | - "my name is alexandria and the customer's name mr sandra"
- "thanks for calling cricket my name is jay may have your name there's calling cricket my name is jay i have your name mm hmm"
- "your mother welcome um so if we can't be free go please return your call us you david gonna transfer pending let me do that so do you have any other concerns in other questions thank you for your time once again my name's jay and thank you for"
|
| 1 | - 'my number to verizon because your services just completely black'
- "forty hour reagan oh my my phone number i'm bye"
- "alright thank you so much richard so yes i can see you're calling in today regarding the same thing i'm so sorry and like your phone number so would it be for out actually right now that was like a port process going on"
|
| 2 | - "okay aah i got a text aah actually right i actually saw it right before the text it says it my payment or my bill was due okay at midnight at i solve it but and then not on aah on the second or the last my son my phones not working right now as a standard but then i got a text right after that aah on the same aah you know text from cricket and it says that my own read it to you this is what they're me off this is why didn't pay it because aah actually it's kind of a for the aah this is another my phone went out that's another thing i need to ask you about get another phone i i do pay insurance a phone in screen went out anyway on that no it says yeah you okay can i get the payment it says the current balance is zero dollars your next bill is due on twelve nine twenty two sir i thought i had to the ninth i m s is clear they actually take me that's why aah know your and so i didn't think i had to pay and so i didn't think i had to pay it cut me off and now aah that's where i'm at and i was trying to pay it with my card i have on file the service something's wrong i don't know what's goin' on"
- "just said i i saw that message after so this one when it off but you know i mean like i said is going to be using why it shouldn't be the other one aah saying that aah there was due on the ninth or said my balance is zero dollars and that my next payment is due on the ninth of the of the four two and aah if this year the also said of ninety eight dollars u s usually my david you'll be so i understand why why then they sent me a"
- "you're saying that your bill is not usually ninety eight dollars"
|
| 3 | - "you're saying that your bill is not usually ninety eight dollars"
- "oh boy the extra effort of calling cricket each month or going into a store to pay your bill just ask one to help you in row today auto pay another great way to save with cricket have you received the suspicious text don't before by spam helps done spam texts my forwarding the message the code seven seven which belt spam went in don't don't click that link did you know that using the my cricket app can save you time and make life easier the mycricket app is a fast and secure way to pay your bill in manage your account okay wendy"
- "already no invalid series streaming now an h b o max when you're on our sixty dollar and the minutes plan you can get h b o max with ads included on i three new shows and what is your movies any time anywhere you want after i presented about our sixty dollar unlimited plan today in a case in the next after i pretty town oh oh here the highly into submitted h b o a regional series house of the draft and would you tell the story amount storage area and said two hundred years before the event of game of thrones is matt an h b o max my end you know the trend is will this you on our sixty dollar unlimited plan you can get h b o max with ads included on on a stream actually originally blockbuster movies a new shows anytime anywhere you want that's the representative better sixty dollar unlimited plan today limitations an exclusion supply ask the recording tells oh oh oh did you know that you can save as much is sixty dollars a year you when you enrolled in autopay customers are most popular single line plans receive five dollars off your monthly my bill with autopay you can use a debit or credit card or reloadable prepaid card and with autopay it's easy to update your form a payment it anytime avoid the extra effort of calling cricket each month or going into a store to pay your bill just ask one of our agents to help you in row today autopay on other great way to save with cricket oh oh have you received a suspicion text don't before by spam helps done spam texts my forwarding the message the code seven seven two six which spells all right thank you so much for your patience just in"
|
## Evaluation
### Metrics
| Label | Accuracy |
|:--------|:---------|
| **all** | 0.8571 |
## Uses
### Direct Use for Inference
First install the SetFit library:
```bash
pip install setfit
```
Then you can load this model and run inference.
```python
from setfit import SetFitModel
# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("Jalajkx/all_mpnetcric-setfit-model")
# Run inference
preds = model("hi welcome to cricket nation thanks for your patience this call may i have your name")
```
## Training Details
### Training Set Metrics
| Training set | Min | Median | Max |
|:-------------|:----|:--------|:----|
| Word count | 5 | 64.6774 | 312 |
| Label | Training Sample Count |
|:------|:----------------------|
| 0 | 7 |
| 1 | 12 |
| 2 | 3 |
| 3 | 4 |
| 4 | 19 |
| 5 | 13 |
| 6 | 4 |
### Training Hyperparameters
- batch_size: (4, 4)
- num_epochs: (1, 1)
- max_steps: -1
- sampling_strategy: oversampling
- num_iterations: 25
- body_learning_rate: (2e-05, 2e-05)
- head_learning_rate: 2e-05
- loss: CosineSimilarityLoss
- distance_metric: cosine_distance
- margin: 0.25
- end_to_end: False
- use_amp: False
- warmup_proportion: 0.1
- seed: 42
- eval_max_steps: -1
- load_best_model_at_end: False
### Training Results
| Epoch | Step | Training Loss | Validation Loss |
|:------:|:----:|:-------------:|:---------------:|
| 0.0013 | 1 | 0.2177 | - |
| 0.0645 | 50 | 0.092 | - |
| 0.1290 | 100 | 0.0167 | - |
| 0.1935 | 150 | 0.1883 | - |
| 0.2581 | 200 | 0.3168 | - |
| 0.3226 | 250 | 0.0372 | - |
| 0.3871 | 300 | 0.0253 | - |
| 0.4516 | 350 | 0.2565 | - |
| 0.5161 | 400 | 0.0096 | - |
| 0.5806 | 450 | 0.0957 | - |
| 0.6452 | 500 | 0.001 | - |
| 0.7097 | 550 | 0.0021 | - |
| 0.7742 | 600 | 0.2043 | - |
| 0.8387 | 650 | 0.0042 | - |
| 0.9032 | 700 | 0.001 | - |
| 0.9677 | 750 | 0.0788 | - |
### Framework Versions
- Python: 3.10.13
- SetFit: 1.0.1
- Sentence Transformers: 2.2.2
- Transformers: 4.36.1
- PyTorch: 2.0.1
- Datasets: 2.15.0
- Tokenizers: 0.15.0
## Citation
### BibTeX
```bibtex
@article{https://doi.org/10.48550/arxiv.2209.11055,
doi = {10.48550/ARXIV.2209.11055},
url = {https://arxiv.org/abs/2209.11055},
author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
title = {Efficient Few-Shot Learning Without Prompts},
publisher = {arXiv},
year = {2022},
copyright = {Creative Commons Attribution 4.0 International}
}
```