metadata
library_name: setfit
tags:
- setfit
- absa
- sentence-transformers
- text-classification
- generated_from_setfit_trainer
base_model: BAAI/bge-small-en-v1.5
metrics:
- accuracy
widget:
- text: >-
to be a nightmare, said retired:The Type 096s are going to be a nightmare,
said retired submariner and naval technical intelligence analyst
Christopher Carlson, one of the researchers.
- text: >-
census as an independent exercise.:In fact, the government of Bihar has
recently taken up the caste census as an independent exercise.
- text: >-
to Moscow's Improved Akula boats.:Carlson told Reuters he did not believe
China had obtained Russia's 'crown jewels' - its very latest technology -
but would be producing a submarine stealthy enough to compare to Moscow's
Improved Akula boats.
- text: >-
staging fully armed nuclear deterrence patrols with its older:The Chinese
navy is routinely staging fully armed nuclear deterrence patrols with its
older Type 094 boats out of Hainan Island in the South China Sea, the
Pentagon said in November, much like patrols operated for years by the
United States, Britain, Russia, and France.
- text: >-
Sanjeev Chopra is a former:Sanjeev Chopra is a former IAS officer and
Festival Director of Valley of Words.
pipeline_tag: text-classification
inference: false
model-index:
- name: SetFit Polarity Model with BAAI/bge-small-en-v1.5
results:
- task:
type: text-classification
name: Text Classification
dataset:
name: Unknown
type: unknown
split: test
metrics:
- type: accuracy
value: 0.8387096774193549
name: Accuracy
SetFit Polarity Model with BAAI/bge-small-en-v1.5
This is a SetFit model that can be used for Aspect Based Sentiment Analysis (ABSA). This SetFit model uses BAAI/bge-small-en-v1.5 as the Sentence Transformer embedding model. A LogisticRegression instance is used for classification. In particular, this model is in charge of classifying aspect polarities.
The model has been trained using an efficient few-shot learning technique that involves:
- Fine-tuning a Sentence Transformer with contrastive learning.
- Training a classification head with features from the fine-tuned Sentence Transformer.
This model was trained within the context of a larger system for ABSA, which looks like so:
- Use a spaCy model to select possible aspect span candidates.
- Use a SetFit model to filter these possible aspect span candidates.
- Use this SetFit model to classify the filtered aspect span candidates.
Model Details
Model Description
- Model Type: SetFit
- Sentence Transformer body: BAAI/bge-small-en-v1.5
- Classification head: a LogisticRegression instance
- spaCy Model: en_core_web_lg
- SetFitABSA Aspect Model: asadnaqvi/setfitabsa-aspect
- SetFitABSA Polarity Model: asadnaqvi/setfitabsa-polarity
- Maximum Sequence Length: 512 tokens
- Number of Classes: 4 classes
Model Sources
- Repository: SetFit on GitHub
- Paper: Efficient Few-Shot Learning Without Prompts
- Blogpost: SetFit: Efficient Few-Shot Learning Without Prompts
Model Labels
| Label | Examples |
|---|---|
| Informative |
|
| Negative |
|
| Positive |
|
| Ambivalent |
|
Evaluation
Metrics
| Label | Accuracy |
|---|---|
| all | 0.8387 |
Uses
Direct Use for Inference
First install the SetFit library:
pip install setfit
Then you can load this model and run inference.
from setfit import AbsaModel
# Download from the 🤗 Hub
model = AbsaModel.from_pretrained(
"asadnaqvi/setfitabsa-aspect",
"asadnaqvi/setfitabsa-polarity",
)
# Run inference
preds = model("The food was great, but the venue is just way too busy.")
Training Details
Training Set Metrics
| Training set | Min | Median | Max |
|---|---|---|---|
| Word count | 11 | 27.7071 | 45 |
| Label | Training Sample Count |
|---|---|
| Ambivalent | 1 |
| Informative | 73 |
| Negative | 20 |
| Positive | 5 |
Training Hyperparameters
- batch_size: (128, 128)
- num_epochs: (5, 5)
- max_steps: -1
- sampling_strategy: oversampling
- body_learning_rate: (2e-05, 1e-05)
- head_learning_rate: 0.01
- loss: CosineSimilarityLoss
- distance_metric: cosine_distance
- margin: 0.25
- end_to_end: False
- use_amp: True
- warmup_proportion: 0.1
- seed: 42
- eval_max_steps: -1
- load_best_model_at_end: True
Training Results
| Epoch | Step | Training Loss | Validation Loss |
|---|---|---|---|
| 0.0217 | 1 | 0.2604 | - |
| 1.0870 | 50 | 0.0642 | 0.3479 |
| 2.1739 | 100 | 0.0249 | 0.3974 |
| 3.2609 | 150 | 0.0146 | 0.4341 |
| 4.3478 | 200 | 0.0033 | 0.4358 |
- The bold row denotes the saved checkpoint.
Framework Versions
- Python: 3.10.12
- SetFit: 1.0.3
- Sentence Transformers: 2.7.0
- spaCy: 3.7.4
- Transformers: 4.40.1
- PyTorch: 2.2.1+cu121
- Datasets: 2.19.0
- Tokenizers: 0.19.1
Citation
BibTeX
@article{https://doi.org/10.48550/arxiv.2209.11055,
doi = {10.48550/ARXIV.2209.11055},
url = {https://arxiv.org/abs/2209.11055},
author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
title = {Efficient Few-Shot Learning Without Prompts},
publisher = {arXiv},
year = {2022},
copyright = {Creative Commons Attribution 4.0 International}
}