davidadamczyk/setfit-model-5

SetFit with sentence-transformers/all-mpnet-base-v2

This is a SetFit model that can be used for Text Classification. This SetFit model uses sentence-transformers/all-mpnet-base-v2 as the Sentence Transformer embedding model. A LogisticRegression instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

1. Fine-tuning a Sentence Transformer with contrastive learning.
2. Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

• Model Type: SetFit
• Sentence Transformer body: sentence-transformers/all-mpnet-base-v2
• Classification head: a LogisticRegression instance
• Maximum Sequence Length: 384 tokens
• Number of Classes: 2 classes

Model Sources

• Repository: SetFit on GitHub
• Paper: Efficient Few-Shot Learning Without Prompts
• Blogpost: SetFit: Efficient Few-Shot Learning Without Prompts

Model Labels

Label	Examples
yes	'Jacinda Ardern is stepping out because her approval rating is below 30%. She has become an ineffective prime minister due in part to rising crime, social inequality, and lingering economic and societal effects of the COVID lockdowns. Workplace burnout is real. But so, apparently, is journalists desire to fit square pegs in round holes. . .\n' "Feeling very sad that Jacinda Ardern, a great liberal leader of her country, is stepping down from her five years as New Zealand's Prime Minister. Her courage was inspirational to women (and men) in her country and everywhere. Would that we had American leaders who would know when they should step aside. ! Stepping down from the burden of political leadership in NZ is an example of Ms. Ardern's great character. Jacinda Ardern will always shine through in the memory of her country!\n" 'Jacinda Ardern is the very definition of public service, and a rare world leader who combined great strength with great empathy. Her decisiveness when protecting her countrymen and women from Covid and in the aftermath of the Christchurch massacre showed what real leadership looks like. And of course she would prioritize the needs of New Zealanders over her premiership if she felt that she had not enough “reserves in the tank” to be as effective as she wished- she’s that sort of politician/public servant. I only hope that with time to recover from her grueling years as prime minister, she will return to the international stage and use her extraordinary presence and talents in a role worthy of her.\n'
no	'Are you not aware of the fact that the people who could have arrested them were overwhelmingly outnumbered with many injuries? Or worse dead. Trump declined to call in the National Guard or other security. Rewatch the video and explain exactly how the insurrectionists could have been arrested on the spot! They are being brought to justice with strong evidence against them.\n' "Frau Greta Absolutely!If there are no consequences ( I mean, we drum this point into our kids about responsibilities and consequences) then what would ever keep future Presidents from willfully breaking the law? The message would be that attaining the Presidency is an automatic 'Get Out of Jail Free' card for life.\n" 'Two candidates trying to outdo each other in their promotion of dismantling of democracy. Two examples of the business model of the modern GOP. One ingratiating themselves with smarmy displays of obeisance the other with industrial strength vitriol. Two loud messages of alarm for the nation.\n'

Evaluation

Metrics

Label	Accuracy
all	1.0

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("davidadamczyk/setfit-model-5")
# Run inference
preds = model("Sure! Support it 100 percent. Good opportunity to watch a president follow the law and accept consequences rather that whine and complain like a toddler.
")

Training Details

Training Set Metrics

Training set	Min	Median	Max
Word count	16	90.75	249

Label	Training Sample Count
no	18
yes	22

Training Hyperparameters

• batch_size: (16, 16)
• num_epochs: (1, 1)
• max_steps: -1
• sampling_strategy: oversampling
• num_iterations: 120
• body_learning_rate: (2e-05, 2e-05)
• head_learning_rate: 2e-05
• loss: CosineSimilarityLoss
• distance_metric: cosine_distance
• margin: 0.25
• end_to_end: False
• use_amp: False
• warmup_proportion: 0.1
• l2_weight: 0.01
• seed: 42
• eval_max_steps: -1
• load_best_model_at_end: False

Training Results

Epoch	Step	Training Loss	Validation Loss
0.0017	1	0.3081	-
0.0833	50	0.1044	-
0.1667	100	0.001	-
0.25	150	0.0003	-
0.3333	200	0.0002	-
0.4167	250	0.0002	-
0.5	300	0.0001	-
0.5833	350	0.0001	-
0.6667	400	0.0001	-
0.75	450	0.0001	-
0.8333	500	0.0001	-
0.9167	550	0.0001	-
1.0	600	0.0001	-

Framework Versions

• Python: 3.10.13
• SetFit: 1.1.0
• Sentence Transformers: 3.0.1
• Transformers: 4.45.2
• PyTorch: 2.4.0+cu124
• Datasets: 2.21.0
• Tokenizers: 0.20.0

Citation

BibTeX

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}