Hugging Face's logo

anismahmahi
/
G2-multilabel-setfit-model

Text Classification
setfit
Safetensors
sentence-transformers
mpnet
generated_from_setfit_trainer
Eval Results
Model card Files Community
G2-multilabel-setfit-model / README.md
anismahmahi's picture
anismahmahi
Add SetFit model
73535a9
preview code
|
raw
history blame contribute delete
9.97 kB
metadata
library_name: setfit
tags:
  - setfit
  - sentence-transformers
  - text-classification
  - generated_from_setfit_trainer
metrics:
  - accuracy
widget:
  - text: |
      It was a jihad training camp.
  - text: >
      Batten echoed that sentiment saying, “Tommy Robinson is a political
      prisoner."
  - text: >
      Failing to answer, Ellison tried to move from person to person, allowing
      his minions to try and provide cover for him, similar to that of Maxine
      Waters, but there was no "member's only" elevator to flee into.
  - text: >
      More details about the horrid compound could be revealed Wednesday when
      the five adults arrested from the site make their first court appearances.
  - text: |
      Black Death Warning: The Plague Is Impossible To Eradicate
pipeline_tag: text-classification
inference: false
base_model: sentence-transformers/paraphrase-mpnet-base-v2
model-index:
  - name: SetFit with sentence-transformers/paraphrase-mpnet-base-v2
    results:
      - task:
          type: text-classification
          name: Text Classification
        dataset:
          name: Unknown
          type: unknown
          split: test
        metrics:
          - type: accuracy
            value: 0.5849056603773585
            name: Accuracy

SetFit with sentence-transformers/paraphrase-mpnet-base-v2

This is a SetFit model that can be used for Text Classification. This SetFit model uses sentence-transformers/paraphrase-mpnet-base-v2 as the Sentence Transformer embedding model. A OneVsRestClassifier instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

  1. Fine-tuning a Sentence Transformer with contrastive learning.
  2. Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

Model Sources

Evaluation

Metrics

Label Accuracy
all 0.5849

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("anismahmahi/G2-multilabel-setfit-model")
# Run inference
preds = model("It was a jihad training camp.
")

Training Details

Training Set Metrics

Training set Min Median Max
Word count 1 26.6518 129

Training Hyperparameters

  • batch_size: (16, 16)
  • num_epochs: (2, 2)
  • max_steps: -1
  • sampling_strategy: oversampling
  • num_iterations: 10
  • body_learning_rate: (2e-05, 1e-05)
  • head_learning_rate: 0.01
  • loss: CosineSimilarityLoss
  • distance_metric: cosine_distance
  • margin: 0.25
  • end_to_end: False
  • use_amp: False
  • warmup_proportion: 0.1
  • seed: 42
  • eval_max_steps: -1
  • load_best_model_at_end: True

Training Results

Epoch Step Training Loss Validation Loss
0.0006 1 0.3905 -
0.0275 50 0.2239 -
0.0550 100 0.2359 -
0.0826 150 0.2443 -
0.1101 200 0.2495 -
0.1376 250 0.2498 -
0.1651 300 0.116 -
0.1926 350 0.1672 -
0.2201 400 0.1281 -
0.2477 450 0.139 -
0.2752 500 0.0615 -
0.3027 550 0.0972 -
0.3302 600 0.0851 -
0.3577 650 0.1769 -
0.3853 700 0.1673 -
0.4128 750 0.0615 -
0.4403 800 0.1232 -
0.4678 850 0.0094 -
0.4953 900 0.0135 -
0.5228 950 0.0107 -
0.5504 1000 0.1137 -
0.5779 1050 0.0173 -
0.6054 1100 0.0573 -
0.6329 1150 0.0115 -
0.6604 1200 0.0374 -
0.6879 1250 0.0231 -
0.7155 1300 0.0392 -
0.7430 1350 0.0754 -
0.7705 1400 0.007 -
0.7980 1450 0.0138 -
0.8255 1500 0.0569 -
0.8531 1550 0.0971 -
0.8806 1600 0.1052 -
0.9081 1650 0.0084 -
0.9356 1700 0.0859 -
0.9631 1750 0.0081 -
0.9906 1800 0.0362 -
1.0 1817 - 0.2354
1.0182 1850 0.0429 -
1.0457 1900 0.056 -
1.0732 1950 0.0098 -
1.1007 2000 0.002 -
1.1282 2050 0.0892 -
1.1558 2100 0.0557 -
1.1833 2150 0.001 -
1.2108 2200 0.0125 -
1.2383 2250 0.0152 -
1.2658 2300 0.0202 -
1.2933 2350 0.0593 -
1.3209 2400 0.007 -
1.3484 2450 0.014 -
1.3759 2500 0.003 -
1.4034 2550 0.0012 -
1.4309 2600 0.0139 -
1.4584 2650 0.0149 -
1.4860 2700 0.002 -
1.5135 2750 0.009 -
1.5410 2800 0.0066 -
1.5685 2850 0.0173 -
1.5960 2900 0.0052 -
1.6236 2950 0.0039 -
1.6511 3000 0.0042 -
1.6786 3050 0.0339 -
1.7061 3100 0.001 -
1.7336 3150 0.0005 -
1.7611 3200 0.0049 -
1.7887 3250 0.01 -
1.8162 3300 0.0815 -
1.8437 3350 0.0227 -
1.8712 3400 0.005 -
1.8987 3450 0.0053 -
1.9263 3500 0.0152 -
1.9538 3550 0.0155 -
1.9813 3600 0.0182 -
2.0 3634 - 0.2266
  • The bold row denotes the saved checkpoint.

Framework Versions

  • Python: 3.10.12
  • SetFit: 1.0.1
  • Sentence Transformers: 2.2.2
  • Transformers: 4.35.2
  • PyTorch: 2.1.0+cu121
  • Datasets: 2.16.1
  • Tokenizers: 0.15.0

Citation

BibTeX 

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}