Add SetFit model

1_Pooling/config.json

@@ -0,0 +1,10 @@
+{
+  "word_embedding_dimension": 384,
+  "pooling_mode_cls_token": true,
+  "pooling_mode_mean_tokens": false,
+  "pooling_mode_max_tokens": false,
+  "pooling_mode_mean_sqrt_len_tokens": false,
+  "pooling_mode_weightedmean_tokens": false,
+  "pooling_mode_lasttoken": false,
+  "include_prompt": true
+}

README.md

@@ -0,0 +1,273 @@
+---
+base_model: BAAI/bge-small-en-v1.5
+library_name: setfit
+metrics:
+- accuracy
+pipeline_tag: text-classification
+tags:
+- setfit
+- sentence-transformers
+- text-classification
+- generated_from_setfit_trainer
+widget:
+- text: dont trust it
+- text: 'works and our AV guys love it people show up with laptops and need to connect
+    plus you can have a secondary monitor as an output we use it for PowerPoint '
+- text: 'I have used Quicken since Microsoft abandoned MSMoney On a Windows PC Sick
+    of the PC crashing freezing fluttering and otherwise giving me the finger I bought
+    a MAC No freezing crashing or security issues Even runs most PC software But not
+    Quicken Just something called Quicken Essentials made for people who don t bank
+    on line don t invest don t have options or IRAs or k accounts In other words made
+    for the folk who buy Lotus for Dummies So I make do with a PC Laptop for accounting
+    using the LAN of my MAC to download and have on it Turbotax as well all the while
+    cursing the Intuit penchant for outdated technology '
+- text: I gave this a this year because the CD just plain flat out didn t work I tried
+    mutliple PCs all with the same resul Please insert a CD Dummy me didn t try the
+    CD until the day return policy had expired so there was no way to return it for
+    a refund I called Intuit and luckily they provided me with a downloadable copy
+    via their site Intuit seemed pretty aware of the problem as they didn t even request
+    the CD be sent to them I should get a refund for all the hassle I went through
+    ha ha
+- text: 'I love TurboTax We use it to prepare our household taxes every year There
+    is a table on the back of every box to help you pick which version you need It
+    has been accurate in my experience When I was young I could get by with a EZ which
+    is equivalent to TurboTax s free software As my career progressed I graduated
+    to TurboTax Basic When I married our combined assets bumped us into Deluxe and
+    then Premier We don t own a business so we may never need Home Business Prior
+    to this I had never revisited Basic I was curious to experience how much I was
+    gaining from using Premier Without going into too much detail the difference is
+    night and day I think they sit too far apart in the gamut for an honest comparison
+    like comparing a Corolla to an Avalon But it is clear that our family will never
+    get by with Basic Thankfully this was provided to me free of charge under the
+    Vine program but otherwise it would have been wasted I ll stick with Premier BOTTOM
+    LINE TurboTax is wonderful but you should follow the advice on the back of the
+    box Don t skimp Buy the version that s right for you Don t be intimidated by the
+    cost You can write off the cost of the software as Tax Prep '
+inference: true
+---
+
+# SetFit with BAAI/bge-small-en-v1.5
+
+This is a [SetFit](https://github.com/huggingface/setfit) model that can be used for Text Classification. This SetFit model uses [BAAI/bge-small-en-v1.5](https://huggingface.co/BAAI/bge-small-en-v1.5) as the Sentence Transformer embedding model. A [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance is used for classification.
+
+The model has been trained using an efficient few-shot learning technique that involves:
+
+1. Fine-tuning a [Sentence Transformer](https://www.sbert.net) with contrastive learning.
+2. Training a classification head with features from the fine-tuned Sentence Transformer.
+
+## Model Details
+
+### Model Description
+- **Model Type:** SetFit
+- **Sentence Transformer body:** [BAAI/bge-small-en-v1.5](https://huggingface.co/BAAI/bge-small-en-v1.5)
+- **Classification head:** a [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance
+- **Maximum Sequence Length:** 512 tokens
+- **Number of Classes:** 2 classes
+<!-- - **Training Dataset:** [Unknown](https://huggingface.co/datasets/unknown) -->
+<!-- - **Language:** Unknown -->
+<!-- - **License:** Unknown -->
+
+### Model Sources
+
+- **Repository:** [SetFit on GitHub](https://github.com/huggingface/setfit)
+- **Paper:** [Efficient Few-Shot Learning Without Prompts](https://arxiv.org/abs/2209.11055)
+- **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit)
+
+### Model Labels
+| Label | Examples                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
+|:------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| 0     | <ul><li>'Been using this excellent product for years don t ever try and do income taxes without it '</li><li>'Use kaspersky every year best product around Will use no other product best prosit I have seen on the market'</li><li>'I ve used Norton before and various free anti virus and with a professional version you get a more comprehensive set of security options that quietly takes care of business in the back ground There is a peace of mind factor that a professional version gives you and for the less than tech savvy it s a bit more idiot proof than a bare bones free ware I have no problem with free ware as my computing needs are pretty simple but a pro version is very nice and this is pretty cheap for the year long comfort of install it and then pretty much forget about it security I got this current product via the Vine but I have bought the professional Norton for the two years running previously when it has been on sale I have multiple computers so the license is handy and I do tend to use all three For the most part Norton is comfortable and user friendly especially if you aren t overly expert with using software '</li></ul>                                                                                                                                                                                                                                                                                                                                          |
+| 1     | <ul><li>'I have use Quicken for over years and I can t believe how cumbersome and poorly conceived this version is compared to past versions The main page is useless and you now have to open multiple windows to get the information you need then you have to close all the windows you opened to get to the next account When looking at a performance page of your investment accounts you get a pie chart instead of a bar graph What good is a pie chart when you are looking at performance data over a specific time range I thought the purpose of newer versions was to improve the existing version and not regress If Microsoft still had a financial program I would be forced to migrate to another program Intuit needs to change it s company name because this program is not intuitive It is ill conceived and makes for a frustrating experience '</li><li>'Would not install activation code not accepted Returned it '</li><li>'I installed this over Norton which I have used and had no problems with My computer slowed to a crawl NAV ate all my computer s resources Activation is a problem and so is its updating proceedures I uninstalled it after it just plain was not working There are still remnents of it on my machine that will not go away I bought Zone Alarm Security Suite ZA Suite is great uses very little resources and my computer is now speedy again Norton is totally overgrown and needs to be rewritten from the source code I will never use a Norton Product again '</li></ul> |
+
+## Uses
+
+### Direct Use for Inference
+
+First install the SetFit library:
+
+```bash
+pip install setfit
+```
+
+Then you can load this model and run inference.
+
+```python
+from setfit import SetFitModel
+
+# Download from the 🤗 Hub
+model = SetFitModel.from_pretrained("selina09/yt_setfit2")
+# Run inference
+preds = model("dont trust it")
+```
+
+<!--
+### Downstream Use
+
+*List how someone could finetune this model on their own dataset.*
+-->
+
+<!--
+### Out-of-Scope Use
+
+*List how the model may foreseeably be misused and address what users ought not to do with the model.*
+-->
+
+<!--
+## Bias, Risks and Limitations
+
+*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
+-->
+
+<!--
+### Recommendations
+
+*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
+-->
+
+## Training Details
+
+### Training Set Metrics
+| Training set | Min | Median  | Max |
+|:-------------|:----|:--------|:----|
+| Word count   | 1   | 93.9133 | 364 |
+
+| Label | Training Sample Count |
+|:------|:----------------------|
+| 0     | 75                    |
+| 1     | 75                    |
+
+### Training Hyperparameters
+- batch_size: (32, 32)
+- num_epochs: (10, 10)
+- max_steps: -1
+- sampling_strategy: oversampling
+- body_learning_rate: (2e-05, 1e-05)
+- head_learning_rate: 0.01
+- loss: CosineSimilarityLoss
+- distance_metric: cosine_distance
+- margin: 0.25
+- end_to_end: False
+- use_amp: False
+- warmup_proportion: 0.1
+- seed: 42
+- eval_max_steps: -1
+- load_best_model_at_end: False
+
+### Training Results
+| Epoch  | Step | Training Loss | Validation Loss |
+|:------:|:----:|:-------------:|:---------------:|
+| 0.0028 | 1    | 0.2613        | -               |
+| 0.1401 | 50   | 0.239         | -               |
+| 0.2801 | 100  | 0.2175        | -               |
+| 0.4202 | 150  | 0.2015        | -               |
+| 0.5602 | 200  | 0.0628        | -               |
+| 0.7003 | 250  | 0.0534        | -               |
+| 0.8403 | 300  | 0.0163        | -               |
+| 0.9804 | 350  | 0.0105        | -               |
+| 1.1204 | 400  | 0.0259        | -               |
+| 1.2605 | 450  | 0.0024        | -               |
+| 1.4006 | 500  | 0.0013        | -               |
+| 1.5406 | 550  | 0.0196        | -               |
+| 1.6807 | 600  | 0.0157        | -               |
+| 1.8207 | 650  | 0.0184        | -               |
+| 1.9608 | 700  | 0.0159        | -               |
+| 2.1008 | 750  | 0.0062        | -               |
+| 2.2409 | 800  | 0.0179        | -               |
+| 2.3810 | 850  | 0.0165        | -               |
+| 2.5210 | 900  | 0.0092        | -               |
+| 2.6611 | 950  | 0.0299        | -               |
+| 2.8011 | 1000 | 0.0071        | -               |
+| 2.9412 | 1050 | 0.0115        | -               |
+| 3.0812 | 1100 | 0.0007        | -               |
+| 3.2213 | 1150 | 0.0248        | -               |
+| 3.3613 | 1200 | 0.0007        | -               |
+| 3.5014 | 1250 | 0.0096        | -               |
+| 3.6415 | 1300 | 0.0091        | -               |
+| 3.7815 | 1350 | 0.0007        | -               |
+| 3.9216 | 1400 | 0.0255        | -               |
+| 4.0616 | 1450 | 0.0065        | -               |
+| 4.2017 | 1500 | 0.0178        | -               |
+| 4.3417 | 1550 | 0.0168        | -               |
+| 4.4818 | 1600 | 0.0161        | -               |
+| 4.6218 | 1650 | 0.0093        | -               |
+| 4.7619 | 1700 | 0.0337        | -               |
+| 4.9020 | 1750 | 0.0148        | -               |
+| 5.0420 | 1800 | 0.0082        | -               |
+| 5.1821 | 1850 | 0.023         | -               |
+| 5.3221 | 1900 | 0.0185        | -               |
+| 5.4622 | 1950 | 0.0155        | -               |
+| 5.6022 | 2000 | 0.0176        | -               |
+| 5.7423 | 2050 | 0.0004        | -               |
+| 5.8824 | 2100 | 0.0221        | -               |
+| 6.0224 | 2150 | 0.0004        | -               |
+| 6.1625 | 2200 | 0.0045        | -               |
+| 6.3025 | 2250 | 0.0004        | -               |
+| 6.4426 | 2300 | 0.0081        | -               |
+| 6.5826 | 2350 | 0.0089        | -               |
+| 6.7227 | 2400 | 0.0091        | -               |
+| 6.8627 | 2450 | 0.0004        | -               |
+| 7.0028 | 2500 | 0.0238        | -               |
+| 7.1429 | 2550 | 0.0056        | -               |
+| 7.2829 | 2600 | 0.0175        | -               |
+| 7.4230 | 2650 | 0.0088        | -               |
+| 7.5630 | 2700 | 0.0383        | -               |
+| 7.7031 | 2750 | 0.0356        | -               |
+| 7.8431 | 2800 | 0.0004        | -               |
+| 7.9832 | 2850 | 0.0231        | -               |
+| 8.1232 | 2900 | 0.0292        | -               |
+| 8.2633 | 2950 | 0.0384        | -               |
+| 8.4034 | 3000 | 0.0004        | -               |
+| 8.5434 | 3050 | 0.0091        | -               |
+| 8.6835 | 3100 | 0.0079        | -               |
+| 8.8235 | 3150 | 0.0298        | -               |
+| 8.9636 | 3200 | 0.0083        | -               |
+| 9.1036 | 3250 | 0.0004        | -               |
+| 9.2437 | 3300 | 0.0003        | -               |
+| 9.3838 | 3350 | 0.0312        | -               |
+| 9.5238 | 3400 | 0.0157        | -               |
+| 9.6639 | 3450 | 0.0003        | -               |
+| 9.8039 | 3500 | 0.0306        | -               |
+| 9.9440 | 3550 | 0.0084        | -               |
+
+### Framework Versions
+- Python: 3.10.12
+- SetFit: 1.0.3
+- Sentence Transformers: 3.0.1
+- Transformers: 4.40.2
+- PyTorch: 2.4.0+cu121
+- Datasets: 2.21.0
+- Tokenizers: 0.19.1
+
+## Citation
+
+### BibTeX
+```bibtex
+@article{https://doi.org/10.48550/arxiv.2209.11055,
+    doi = {10.48550/ARXIV.2209.11055},
+    url = {https://arxiv.org/abs/2209.11055},
+    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
+    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
+    title = {Efficient Few-Shot Learning Without Prompts},
+    publisher = {arXiv},
+    year = {2022},
+    copyright = {Creative Commons Attribution 4.0 International}
+}
+```
+
+<!--
+## Glossary
+
+*Clearly define terms in order to be accessible across audiences.*
+-->
+
+<!--
+## Model Card Authors
+
+*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
+-->
+
+<!--
+## Model Card Contact
+
+*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
+-->

config.json

@@ -0,0 +1,31 @@
+{
+  "_name_or_path": "BAAI/bge-small-en-v1.5",
+  "architectures": [
+    "BertModel"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "classifier_dropout": null,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 384,
+  "id2label": {
+    "0": "LABEL_0"
+  },
+  "initializer_range": 0.02,
+  "intermediate_size": 1536,
+  "label2id": {
+    "LABEL_0": 0
+  },
+  "layer_norm_eps": 1e-12,
+  "max_position_embeddings": 512,
+  "model_type": "bert",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "pad_token_id": 0,
+  "position_embedding_type": "absolute",
+  "torch_dtype": "float32",
+  "transformers_version": "4.40.2",
+  "type_vocab_size": 2,
+  "use_cache": true,
+  "vocab_size": 30522
+}

config_sentence_transformers.json

@@ -0,0 +1,10 @@
+{
+  "__version__": {
+    "sentence_transformers": "3.0.1",
+    "transformers": "4.40.2",
+    "pytorch": "2.4.0+cu121"
+  },
+  "prompts": {},
+  "default_prompt_name": null,
+  "similarity_fn_name": null
+}

config_setfit.json

@@ -0,0 +1,4 @@
+{
+  "normalize_embeddings": false,
+  "labels": null
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ad3e4e25e365a29a87a9bf8ca9072c0eec4e7015f05e6bcd1d4d7cf90bb2fc57
+size 133462128

model_head.pkl

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c8dfd15cfd4622f2940839f128306c8bde66a365633aef9151119e6cc2c9ffc9
+size 3935

modules.json

@@ -0,0 +1,20 @@
+[
+  {
+    "idx": 0,
+    "name": "0",
+    "path": "",
+    "type": "sentence_transformers.models.Transformer"
+  },
+  {
+    "idx": 1,
+    "name": "1",
+    "path": "1_Pooling",
+    "type": "sentence_transformers.models.Pooling"
+  },
+  {
+    "idx": 2,
+    "name": "2",
+    "path": "2_Normalize",
+    "type": "sentence_transformers.models.Normalize"
+  }
+]

sentence_bert_config.json

@@ -0,0 +1,4 @@
+{
+  "max_seq_length": 512,
+  "do_lower_case": true
+}

special_tokens_map.json

@@ -0,0 +1,37 @@
+{
+  "cls_token": {
+    "content": "[CLS]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "[MASK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "[PAD]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "sep_token": {
+    "content": "[SEP]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "[UNK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer_config.json

@@ -0,0 +1,57 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "101": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "102": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "103": {
+      "content": "[MASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "[CLS]",
+  "do_basic_tokenize": true,
+  "do_lower_case": true,
+  "mask_token": "[MASK]",
+  "model_max_length": 512,
+  "never_split": null,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "BertTokenizer",
+  "unk_token": "[UNK]"
+}