SetFit with sentence-transformers/paraphrase-mpnet-base-v2

This is a SetFit model that can be used for Text Classification. This SetFit model uses sentence-transformers/paraphrase-mpnet-base-v2 as the Sentence Transformer embedding model. A SetFitHead instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

Fine-tuning a Sentence Transformer with contrastive learning.
Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

Model Type: SetFit
Sentence Transformer body: sentence-transformers/paraphrase-mpnet-base-v2
Classification head: a SetFitHead instance
Maximum Sequence Length: 512 tokens
Number of Classes: 7 classes

Model Sources

Repository: SetFit on GitHub
Paper: Efficient Few-Shot Learning Without Prompts
Blogpost: SetFit: Efficient Few-Shot Learning Without Prompts

Model Labels

Label	Examples
4	"Ringing does become a bit of an obsession , `admits Stephanie Pattenden , master of the band at St. Mary Abbot and one of England 's best female ringers . "</li><li>'` All this may not be obvious to the public , which is concerned about advances in treatment , but I am convinced this basic research will begin showing results there soon . `` ' 'So the driver started to curse at both of them as if they had been in a plot together to ruin his safe-driving record . '
5	"But is n't the desire for profit the driving force behind those who subscribe to , and advertise in , your paper ? " 'Finally , the committee of discipline of the UEFA has sanctioned with two games the trainer of the team , Jose Mourinho , and with different economic amounts the implied players . ' "In their filing , they referred to the Supreme Court 's 1966 decision overturning Dr. Sheppard 's conviction because of the trial coverage and made it clear they would claim that the new media required new rules . "
1	'We see that the peak of the Spanish selection is the goalkeeper Iker Casillas , who last Saturday against England ( game lost with 0-1 in London ) equaled the record of the ex goalkeeper Andoni Zubizarreta , with 126 international matches . ' 'The one exception to this recent trend was the defeat of 13 of the 52 freshman Republicans brought into office in 1980 by the Reagan revolution and running for re-election in 1982 . ' 'When the light went their way , they went on across the street . '
2	'This is a Johnson-era , Great Society creation that mandates certain government contracts be awarded noncompetitively to minority businesses . ' 'Today , we know that the accumulation of several of these altered genes can initiate a cancer and , then , propel it into a deadly state . `'</li><li>'Let us sum up what we do know about education and about those education reforms that do work and do not work : --` Parental involvement `` is a bad idea . '
3	'Far above in the belfry , the huge bronze bells , mounted on wheels , swing madly through a full 360 degrees , starting and ending , surprisingly , in the inverted , or mouth-up position . ' 'There you have a list of the declared variables . ' 'Medical scientists are starting to uncover a handful of genes which , if damaged , unleash the chaotic growth of cells that characterizes cancer . '
0	'They belong to a group of 15 ringers -- including two octogenarians and four youngsters in training -- who drive every Sunday from church to church in a sometimes-exhausting effort to keep the bells sounding in the many belfries of East Anglia . ' 'Executes the instructions in a file sequentially . ' 'In quoting from our research you emphasized the high prevalance of mental illness and alcoholism . '
6	'It often is preceded by the development of polyps in the bowel , which in some cases become increasingly malignant in identifiable stages -- progressing from less severe to deadly -- as though a cascade of genetic damage might be occurring . ' 'I always work it that way -- and always at a time when the customer has an alibi . ' 'A person who is born with one defective copy of a suppressor gene , or in whom one copy is damaged early in life , is especially prone to cancer because he need only lose the other copy for a cancer to develop . '

Evaluation

Metrics

Label	Accuracy
all	0.1547

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("HelgeKn/SemEval-multi-class-20")
# Run inference
preds = model("For follow-up treatment , the animal owner can administer the tablets to the dog . ")

Training Details

Training Set Metrics

Training set	Min	Median	Max
Word count	4	26.5857	74

Label	Training Sample Count
0	20
1	20
2	20
3	20
4	20
5	20
6	20

Training Hyperparameters

batch_size: (16, 16)
num_epochs: (2, 2)
max_steps: -1
sampling_strategy: oversampling
num_iterations: 20
body_learning_rate: (2e-05, 2e-05)
head_learning_rate: 2e-05
loss: CosineSimilarityLoss
distance_metric: cosine_distance
margin: 0.25
end_to_end: False
use_amp: False
warmup_proportion: 0.1
seed: 42
eval_max_steps: -1
load_best_model_at_end: False

Training Results

Epoch	Step	Training Loss	Validation Loss
0.0029	1	0.3312	-
0.1429	50	0.264	-
0.2857	100	0.2359	-
0.4286	150	0.2107	-
0.5714	200	0.2034	-
0.7143	250	0.114	-
0.8571	300	0.0381	-
1.0	350	0.0395	-
1.1429	400	0.013	-
1.2857	450	0.0035	-
1.4286	500	0.0028	-
1.5714	550	0.0025	-
1.7143	600	0.002	-
1.8571	650	0.002	-
2.0	700	0.0026	-

Framework Versions

Python: 3.9.13
SetFit: 1.0.1
Sentence Transformers: 2.2.2
Transformers: 4.36.0
PyTorch: 2.1.1+cpu
Datasets: 2.15.0
Tokenizers: 0.15.0

Citation

BibTeX

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}

HelgeKn
/

SemEval-multi-class-20