--- library_name: setfit tags: - setfit - sentence-transformers - text-classification - generated_from_setfit_trainer metrics: - accuracy widget: - text: aku hanya menyukai setiap menit film ini. - text: bioskop orang dalam kondisi terbaiknya. - text: bukan untuk orang yang mudah tersinggung atau mudah tersinggung, ini adalah pemeriksaan yang berani dan berkepanjangan terhadap budaya yang diidolakan, kebencian terhadap diri sendiri, dan politik seksual. - text: itu curang. - text: Meskipun penduduk setempat akan senang melihat situs-situs Cleveland, seluruh dunia akan menikmati komedi bertempo cepat dengan keunikan yang mungkin membuat iri para coen bersaudara yang telah memenangkan penghargaan. pipeline_tag: text-classification inference: true base_model: firqaaa/indo-sentence-bert-base model-index: - name: SetFit with firqaaa/indo-sentence-bert-base results: - task: type: text-classification name: Text Classification dataset: name: Unknown type: unknown split: test metrics: - type: accuracy value: 0.4248868778280543 name: Accuracy --- # SetFit with firqaaa/indo-sentence-bert-base This is a [SetFit](https://github.com/huggingface/setfit) model that can be used for Text Classification. This SetFit model uses [firqaaa/indo-sentence-bert-base](https://huggingface.co/firqaaa/indo-sentence-bert-base) as the Sentence Transformer embedding model. A [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance is used for classification. The model has been trained using an efficient few-shot learning technique that involves: 1. Fine-tuning a [Sentence Transformer](https://www.sbert.net) with contrastive learning. 2. Training a classification head with features from the fine-tuned Sentence Transformer. ## Model Details ### Model Description - **Model Type:** SetFit - **Sentence Transformer body:** [firqaaa/indo-sentence-bert-base](https://huggingface.co/firqaaa/indo-sentence-bert-base) - **Classification head:** a [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance - **Maximum Sequence Length:** 512 tokens - **Number of Classes:** 5 classes ### Model Sources - **Repository:** [SetFit on GitHub](https://github.com/huggingface/setfit) - **Paper:** [Efficient Few-Shot Learning Without Prompts](https://arxiv.org/abs/2209.11055) - **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit) ### Model Labels | Label | Examples | |:---------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | negatif | | | positif | | | sangat negatif | | | netral | | | sangat positif | | ## Evaluation ### Metrics | Label | Accuracy | |:--------|:---------| | **all** | 0.4249 | ## Uses ### Direct Use for Inference First install the SetFit library: ```bash pip install setfit ``` Then you can load this model and run inference. ```python from setfit import SetFitModel # Download from the 🤗 Hub model = SetFitModel.from_pretrained("firqaaa/indo-setfit-bert-base-p2") # Run inference preds = model("itu curang.") ``` ## Training Details ### Training Set Metrics | Training set | Min | Median | Max | |:-------------|:----|:--------|:----| | Word count | 1 | 15.7676 | 46 | | Label | Training Sample Count | |:---------------|:----------------------| | sangat negatif | 500 | | negatif | 500 | | netral | 500 | | positif | 500 | | sangat positif | 500 | ### Training Hyperparameters - batch_size: (128, 128) - num_epochs: (1, 1) - max_steps: -1 - sampling_strategy: oversampling - body_learning_rate: (2e-05, 1e-05) - head_learning_rate: 0.01 - loss: CosineSimilarityLoss - distance_metric: cosine_distance - margin: 0.25 - end_to_end: False - use_amp: False - warmup_proportion: 0.1 - seed: 42 - eval_max_steps: -1 - load_best_model_at_end: True ### Training Results | Epoch | Step | Training Loss | Validation Loss | |:-------:|:---------:|:-------------:|:---------------:| | 0.0000 | 1 | 0.3367 | - | | 0.0013 | 50 | 0.3139 | - | | 0.0026 | 100 | 0.3005 | - | | 0.0038 | 150 | 0.2627 | - | | 0.0051 | 200 | 0.2701 | - | | 0.0064 | 250 | 0.2647 | - | | 0.0077 | 300 | 0.2646 | - | | 0.0090 | 350 | 0.2494 | - | | 0.0102 | 400 | 0.2356 | - | | 0.0115 | 450 | 0.2093 | - | | 0.0128 | 500 | 0.2187 | - | | 0.0141 | 550 | 0.2131 | - | | 0.0154 | 600 | 0.2288 | - | | 0.0166 | 650 | 0.1996 | - | | 0.0179 | 700 | 0.1825 | - | | 0.0192 | 750 | 0.1887 | - | | 0.0205 | 800 | 0.1809 | - | | 0.0218 | 850 | 0.1756 | - | | 0.0230 | 900 | 0.155 | - | | 0.0243 | 950 | 0.1462 | - | | 0.0256 | 1000 | 0.1455 | - | | 0.0269 | 1050 | 0.1547 | - | | 0.0282 | 1100 | 0.0863 | - | | 0.0294 | 1150 | 0.1362 | - | | 0.0307 | 1200 | 0.1096 | - | | 0.0320 | 1250 | 0.0898 | - | | 0.0333 | 1300 | 0.1202 | - | | 0.0346 | 1350 | 0.0916 | - | | 0.0358 | 1400 | 0.0918 | - | | 0.0371 | 1450 | 0.1022 | - | | 0.0384 | 1500 | 0.0518 | - | | 0.0397 | 1550 | 0.0587 | - | | 0.0410 | 1600 | 0.0526 | - | | 0.0422 | 1650 | 0.0461 | - | | 0.0435 | 1700 | 0.0617 | - | | 0.0448 | 1750 | 0.0426 | - | | 0.0461 | 1800 | 0.0347 | - | | 0.0474 | 1850 | 0.0255 | - | | 0.0486 | 1900 | 0.0349 | - | | 0.0499 | 1950 | 0.0121 | - | | 0.0512 | 2000 | 0.0164 | - | | 0.0525 | 2050 | 0.0077 | - | | 0.0538 | 2100 | 0.0084 | - | | 0.0550 | 2150 | 0.006 | - | | 0.0563 | 2200 | 0.0143 | - | | 0.0576 | 2250 | 0.0123 | - | | 0.0589 | 2300 | 0.0154 | - | | 0.0602 | 2350 | 0.0108 | - | | 0.0614 | 2400 | 0.0041 | - | | 0.0627 | 2450 | 0.0048 | - | | 0.0640 | 2500 | 0.0103 | - | | 0.0653 | 2550 | 0.0099 | - | | 0.0666 | 2600 | 0.026 | - | | 0.0678 | 2650 | 0.0095 | - | | 0.0691 | 2700 | 0.0091 | - | | 0.0704 | 2750 | 0.0041 | - | | 0.0717 | 2800 | 0.005 | - | | 0.0730 | 2850 | 0.0024 | - | | 0.0742 | 2900 | 0.0013 | - | | 0.0755 | 2950 | 0.0067 | - | | 0.0768 | 3000 | 0.0009 | - | | 0.0781 | 3050 | 0.0042 | - | | 0.0794 | 3100 | 0.0039 | - | | 0.0806 | 3150 | 0.0023 | - | | 0.0819 | 3200 | 0.0032 | - | | 0.0832 | 3250 | 0.0071 | - | | 0.0845 | 3300 | 0.013 | - | | 0.0858 | 3350 | 0.015 | - | | 0.0870 | 3400 | 0.0013 | - | | 0.0883 | 3450 | 0.0012 | - | | 0.0896 | 3500 | 0.0017 | - | | 0.0909 | 3550 | 0.002 | - | | 0.0922 | 3600 | 0.0247 | - | | 0.0934 | 3650 | 0.0044 | - | | 0.0947 | 3700 | 0.0004 | - | | 0.0960 | 3750 | 0.0031 | - | | 0.0973 | 3800 | 0.0235 | - | | 0.0986 | 3850 | 0.0017 | - | | 0.0998 | 3900 | 0.001 | - | | 0.1011 | 3950 | 0.0065 | - | | 0.1024 | 4000 | 0.0043 | - | | 0.1037 | 4050 | 0.0051 | - | | 0.1050 | 4100 | 0.0009 | - | | 0.1062 | 4150 | 0.0006 | - | | 0.1075 | 4200 | 0.0081 | - | | 0.1088 | 4250 | 0.0005 | - | | 0.1101 | 4300 | 0.0155 | - | | 0.1114 | 4350 | 0.0091 | - | | 0.1126 | 4400 | 0.0187 | - | | 0.1139 | 4450 | 0.0011 | - | | 0.1152 | 4500 | 0.0037 | - | | 0.1165 | 4550 | 0.0033 | - | | 0.1178 | 4600 | 0.0006 | - | | 0.1190 | 4650 | 0.0024 | - | | 0.1203 | 4700 | 0.0008 | - | | 0.1216 | 4750 | 0.0007 | - | | 0.1229 | 4800 | 0.0012 | - | | 0.1242 | 4850 | 0.0113 | - | | 0.1254 | 4900 | 0.0004 | - | | 0.1267 | 4950 | 0.0059 | - | | 0.1280 | 5000 | 0.0004 | - | | 0.1293 | 5050 | 0.001 | - | | 0.1306 | 5100 | 0.0001 | - | | 0.1318 | 5150 | 0.002 | - | | 0.1331 | 5200 | 0.0006 | - | | 0.1344 | 5250 | 0.0007 | - | | 0.1357 | 5300 | 0.0026 | - | | 0.1370 | 5350 | 0.0079 | - | | 0.1382 | 5400 | 0.001 | - | | 0.1395 | 5450 | 0.0065 | - | | 0.1408 | 5500 | 0.0009 | - | | 0.1421 | 5550 | 0.0008 | - | | 0.1434 | 5600 | 0.0003 | - | | 0.1446 | 5650 | 0.0002 | - | | 0.1459 | 5700 | 0.0001 | - | | 0.1472 | 5750 | 0.0027 | - | | 0.1485 | 5800 | 0.0002 | - | | 0.1498 | 5850 | 0.0002 | - | | 0.1510 | 5900 | 0.0003 | - | | 0.1523 | 5950 | 0.0001 | - | | 0.1536 | 6000 | 0.0061 | - | | 0.1549 | 6050 | 0.0066 | - | | 0.1562 | 6100 | 0.0015 | - | | 0.1574 | 6150 | 0.016 | - | | 0.1587 | 6200 | 0.0009 | - | | 0.1600 | 6250 | 0.0062 | - | | 0.1613 | 6300 | 0.0002 | - | | 0.1626 | 6350 | 0.0002 | - | | 0.1638 | 6400 | 0.0002 | - | | 0.1651 | 6450 | 0.0153 | - | | 0.1664 | 6500 | 0.0031 | - | | 0.1677 | 6550 | 0.0003 | - | | 0.1690 | 6600 | 0.0009 | - | | 0.1702 | 6650 | 0.0043 | - | | 0.1715 | 6700 | 0.0007 | - | | 0.1728 | 6750 | 0.0002 | - | | 0.1741 | 6800 | 0.0001 | - | | 0.1754 | 6850 | 0.0003 | - | | 0.1766 | 6900 | 0.0013 | - | | 0.1779 | 6950 | 0.0003 | - | | 0.1792 | 7000 | 0.0002 | - | | 0.1805 | 7050 | 0.0001 | - | | 0.1818 | 7100 | 0.0001 | - | | 0.1830 | 7150 | 0.0001 | - | | 0.1843 | 7200 | 0.0001 | - | | 0.1856 | 7250 | 0.0003 | - | | 0.1869 | 7300 | 0.0001 | - | | 0.1882 | 7350 | 0.0002 | - | | 0.1894 | 7400 | 0.0012 | - | | 0.1907 | 7450 | 0.0001 | - | | 0.1920 | 7500 | 0.0002 | - | | 0.1933 | 7550 | 0.0002 | - | | 0.1946 | 7600 | 0.0003 | - | | 0.1958 | 7650 | 0.0014 | - | | 0.1971 | 7700 | 0.0093 | - | | 0.1984 | 7750 | 0.0001 | - | | 0.1997 | 7800 | 0.0005 | - | | 0.2010 | 7850 | 0.0001 | - | | 0.2022 | 7900 | 0.0001 | - | | 0.2035 | 7950 | 0.0058 | - | | 0.2048 | 8000 | 0.0002 | - | | 0.2061 | 8050 | 0.0001 | - | | 0.2074 | 8100 | 0.0002 | - | | 0.2086 | 8150 | 0.0003 | - | | 0.2099 | 8200 | 0.0003 | - | | 0.2112 | 8250 | 0.0068 | - | | 0.2125 | 8300 | 0.0004 | - | | 0.2138 | 8350 | 0.0002 | - | | 0.2150 | 8400 | 0.0001 | - | | 0.2163 | 8450 | 0.0002 | - | | 0.2176 | 8500 | 0.0001 | - | | 0.2189 | 8550 | 0.0002 | - | | 0.2202 | 8600 | 0.0001 | - | | 0.2214 | 8650 | 0.0001 | - | | 0.2227 | 8700 | 0.0001 | - | | 0.2240 | 8750 | 0.0001 | - | | 0.2253 | 8800 | 0.0001 | - | | 0.2266 | 8850 | 0.0006 | - | | 0.2278 | 8900 | 0.0 | - | | 0.2291 | 8950 | 0.0 | - | | 0.2304 | 9000 | 0.0001 | - | | 0.2317 | 9050 | 0.0 | - | | 0.2330 | 9100 | 0.0001 | - | | 0.2342 | 9150 | 0.0 | - | | 0.2355 | 9200 | 0.0001 | - | | 0.2368 | 9250 | 0.0 | - | | 0.2381 | 9300 | 0.0001 | - | | 0.2394 | 9350 | 0.0001 | - | | 0.2406 | 9400 | 0.0 | - | | 0.2419 | 9450 | 0.0 | - | | 0.2432 | 9500 | 0.0001 | - | | 0.2445 | 9550 | 0.0 | - | | 0.2458 | 9600 | 0.0001 | - | | 0.2470 | 9650 | 0.0001 | - | | 0.2483 | 9700 | 0.003 | - | | 0.2496 | 9750 | 0.0077 | - | | 0.2509 | 9800 | 0.0099 | - | | 0.2522 | 9850 | 0.0223 | - | | 0.2534 | 9900 | 0.0002 | - | | 0.2547 | 9950 | 0.0001 | - | | 0.2560 | 10000 | 0.003 | - | | 0.2573 | 10050 | 0.0118 | - | | 0.2586 | 10100 | 0.0002 | - | | 0.2598 | 10150 | 0.0022 | - | | 0.2611 | 10200 | 0.0001 | - | | 0.2624 | 10250 | 0.0077 | - | | 0.2637 | 10300 | 0.0003 | - | | 0.2650 | 10350 | 0.0 | - | | 0.2662 | 10400 | 0.0074 | - | | 0.2675 | 10450 | 0.0072 | - | | 0.2688 | 10500 | 0.0001 | - | | 0.2701 | 10550 | 0.008 | - | | 0.2714 | 10600 | 0.0001 | - | | 0.2726 | 10650 | 0.0001 | - | | 0.2739 | 10700 | 0.0 | - | | 0.2752 | 10750 | 0.0001 | - | | 0.2765 | 10800 | 0.0074 | - | | 0.2778 | 10850 | 0.0001 | - | | 0.2790 | 10900 | 0.0001 | - | | 0.2803 | 10950 | 0.0003 | - | | 0.2816 | 11000 | 0.0004 | - | | 0.2829 | 11050 | 0.0078 | - | | 0.2842 | 11100 | 0.0 | - | | 0.2854 | 11150 | 0.0001 | - | | 0.2867 | 11200 | 0.0001 | - | | 0.2880 | 11250 | 0.0001 | - | | 0.2893 | 11300 | 0.0 | - | | 0.2906 | 11350 | 0.0001 | - | | 0.2918 | 11400 | 0.0001 | - | | 0.2931 | 11450 | 0.0004 | - | | 0.2944 | 11500 | 0.0002 | - | | 0.2957 | 11550 | 0.0 | - | | 0.2970 | 11600 | 0.0 | - | | 0.2982 | 11650 | 0.0078 | - | | 0.2995 | 11700 | 0.0 | - | | 0.3008 | 11750 | 0.0005 | - | | 0.3021 | 11800 | 0.0001 | - | | 0.3034 | 11850 | 0.0 | - | | 0.3046 | 11900 | 0.0 | - | | 0.3059 | 11950 | 0.0 | - | | 0.3072 | 12000 | 0.0006 | - | | 0.3085 | 12050 | 0.0078 | - | | 0.3098 | 12100 | 0.0001 | - | | 0.3110 | 12150 | 0.0 | - | | 0.3123 | 12200 | 0.0 | - | | 0.3136 | 12250 | 0.0 | - | | 0.3149 | 12300 | 0.0 | - | | 0.3162 | 12350 | 0.0 | - | | 0.3174 | 12400 | 0.0 | - | | 0.3187 | 12450 | 0.0 | - | | 0.3200 | 12500 | 0.0 | - | | 0.3213 | 12550 | 0.0002 | - | | 0.3226 | 12600 | 0.0 | - | | 0.3238 | 12650 | 0.0003 | - | | 0.3251 | 12700 | 0.0001 | - | | 0.3264 | 12750 | 0.0001 | - | | 0.3277 | 12800 | 0.0 | - | | 0.3290 | 12850 | 0.0001 | - | | 0.3302 | 12900 | 0.0001 | - | | 0.3315 | 12950 | 0.0001 | - | | 0.3328 | 13000 | 0.0 | - | | 0.3341 | 13050 | 0.0 | - | | 0.3354 | 13100 | 0.0 | - | | 0.3366 | 13150 | 0.0 | - | | 0.3379 | 13200 | 0.0 | - | | 0.3392 | 13250 | 0.0 | - | | 0.3405 | 13300 | 0.0 | - | | 0.3418 | 13350 | 0.0 | - | | 0.3430 | 13400 | 0.0 | - | | 0.3443 | 13450 | 0.0 | - | | 0.3456 | 13500 | 0.0 | - | | 0.3469 | 13550 | 0.0005 | - | | 0.3482 | 13600 | 0.0 | - | | 0.3494 | 13650 | 0.0 | - | | 0.3507 | 13700 | 0.0 | - | | 0.3520 | 13750 | 0.0 | - | | 0.3533 | 13800 | 0.0011 | - | | 0.3546 | 13850 | 0.0001 | - | | 0.3558 | 13900 | 0.0079 | - | | 0.3571 | 13950 | 0.0001 | - | | 0.3584 | 14000 | 0.0 | - | | 0.3597 | 14050 | 0.0 | - | | 0.3610 | 14100 | 0.0 | - | | 0.3622 | 14150 | 0.0 | - | | 0.3635 | 14200 | 0.0074 | - | | 0.3648 | 14250 | 0.0 | - | | 0.3661 | 14300 | 0.0 | - | | 0.3674 | 14350 | 0.0001 | - | | 0.3686 | 14400 | 0.0 | - | | 0.3699 | 14450 | 0.0001 | - | | 0.3712 | 14500 | 0.0 | - | | 0.3725 | 14550 | 0.0 | - | | 0.3738 | 14600 | 0.0 | - | | 0.3750 | 14650 | 0.0002 | - | | 0.3763 | 14700 | 0.0001 | - | | 0.3776 | 14750 | 0.0 | - | | 0.3789 | 14800 | 0.0001 | - | | 0.3802 | 14850 | 0.0 | - | | 0.3814 | 14900 | 0.0001 | - | | 0.3827 | 14950 | 0.0 | - | | 0.3840 | 15000 | 0.0 | - | | 0.3853 | 15050 | 0.0 | - | | 0.3866 | 15100 | 0.0 | - | | 0.3878 | 15150 | 0.0 | - | | 0.3891 | 15200 | 0.0 | - | | 0.3904 | 15250 | 0.0 | - | | 0.3917 | 15300 | 0.0001 | - | | 0.3930 | 15350 | 0.0 | - | | 0.3942 | 15400 | 0.0 | - | | 0.3955 | 15450 | 0.0 | - | | 0.3968 | 15500 | 0.0 | - | | 0.3981 | 15550 | 0.0 | - | | 0.3994 | 15600 | 0.0 | - | | 0.4006 | 15650 | 0.0 | - | | 0.4019 | 15700 | 0.0 | - | | 0.4032 | 15750 | 0.0001 | - | | 0.4045 | 15800 | 0.0 | - | | 0.4058 | 15850 | 0.0 | - | | 0.4070 | 15900 | 0.0 | - | | 0.4083 | 15950 | 0.0 | - | | 0.4096 | 16000 | 0.0 | - | | 0.4109 | 16050 | 0.0 | - | | 0.4122 | 16100 | 0.0 | - | | 0.4134 | 16150 | 0.0 | - | | 0.4147 | 16200 | 0.0 | - | | 0.4160 | 16250 | 0.0003 | - | | 0.4173 | 16300 | 0.0 | - | | 0.4186 | 16350 | 0.0 | - | | 0.4198 | 16400 | 0.0 | - | | 0.4211 | 16450 | 0.0 | - | | 0.4224 | 16500 | 0.0 | - | | 0.4237 | 16550 | 0.0 | - | | 0.4250 | 16600 | 0.0 | - | | 0.4262 | 16650 | 0.0 | - | | 0.4275 | 16700 | 0.0 | - | | 0.4288 | 16750 | 0.0 | - | | 0.4301 | 16800 | 0.0 | - | | 0.4314 | 16850 | 0.0 | - | | 0.4326 | 16900 | 0.0 | - | | 0.4339 | 16950 | 0.0 | - | | 0.4352 | 17000 | 0.0 | - | | 0.4365 | 17050 | 0.0 | - | | 0.4378 | 17100 | 0.0 | - | | 0.4390 | 17150 | 0.0 | - | | 0.4403 | 17200 | 0.0 | - | | 0.4416 | 17250 | 0.0 | - | | 0.4429 | 17300 | 0.0 | - | | 0.4442 | 17350 | 0.0 | - | | 0.4454 | 17400 | 0.0 | - | | 0.4467 | 17450 | 0.0 | - | | 0.4480 | 17500 | 0.0016 | - | | 0.4493 | 17550 | 0.0 | - | | 0.4506 | 17600 | 0.0 | - | | 0.4518 | 17650 | 0.0 | - | | 0.4531 | 17700 | 0.0 | - | | 0.4544 | 17750 | 0.0 | - | | 0.4557 | 17800 | 0.0 | - | | 0.4570 | 17850 | 0.0 | - | | 0.4582 | 17900 | 0.0 | - | | 0.4595 | 17950 | 0.0068 | - | | 0.4608 | 18000 | 0.0001 | - | | 0.4621 | 18050 | 0.0001 | - | | 0.4634 | 18100 | 0.0001 | - | | 0.4646 | 18150 | 0.0001 | - | | 0.4659 | 18200 | 0.0001 | - | | 0.4672 | 18250 | 0.0 | - | | 0.4685 | 18300 | 0.0 | - | | 0.4698 | 18350 | 0.0001 | - | | 0.4710 | 18400 | 0.0 | - | | 0.4723 | 18450 | 0.0 | - | | 0.4736 | 18500 | 0.0 | - | | 0.4749 | 18550 | 0.0 | - | | 0.4762 | 18600 | 0.0 | - | | 0.4774 | 18650 | 0.0 | - | | 0.4787 | 18700 | 0.0 | - | | 0.4800 | 18750 | 0.0 | - | | 0.4813 | 18800 | 0.0 | - | | 0.4826 | 18850 | 0.0 | - | | 0.4838 | 18900 | 0.0 | - | | 0.4851 | 18950 | 0.0 | - | | 0.4864 | 19000 | 0.0 | - | | 0.4877 | 19050 | 0.0 | - | | 0.4890 | 19100 | 0.0 | - | | 0.4902 | 19150 | 0.0 | - | | 0.4915 | 19200 | 0.0 | - | | 0.4928 | 19250 | 0.0 | - | | 0.4941 | 19300 | 0.0 | - | | 0.4954 | 19350 | 0.0 | - | | 0.4966 | 19400 | 0.0 | - | | 0.4979 | 19450 | 0.0 | - | | 0.4992 | 19500 | 0.0 | - | | 0.5005 | 19550 | 0.0 | - | | 0.5018 | 19600 | 0.0 | - | | 0.5030 | 19650 | 0.0 | - | | 0.5043 | 19700 | 0.0 | - | | 0.5056 | 19750 | 0.0 | - | | 0.5069 | 19800 | 0.0 | - | | 0.5082 | 19850 | 0.0 | - | | 0.5094 | 19900 | 0.0 | - | | 0.5107 | 19950 | 0.0 | - | | 0.5120 | 20000 | 0.0 | - | | 0.5133 | 20050 | 0.0 | - | | 0.5146 | 20100 | 0.0 | - | | 0.5158 | 20150 | 0.0 | - | | 0.5171 | 20200 | 0.0 | - | | 0.5184 | 20250 | 0.0 | - | | 0.5197 | 20300 | 0.0 | - | | 0.5210 | 20350 | 0.0 | - | | 0.5222 | 20400 | 0.0 | - | | 0.5235 | 20450 | 0.0 | - | | 0.5248 | 20500 | 0.0 | - | | 0.5261 | 20550 | 0.0 | - | | 0.5274 | 20600 | 0.0 | - | | 0.5286 | 20650 | 0.0 | - | | 0.5299 | 20700 | 0.0 | - | | 0.5312 | 20750 | 0.0 | - | | 0.5325 | 20800 | 0.0 | - | | 0.5338 | 20850 | 0.0 | - | | 0.5350 | 20900 | 0.0 | - | | 0.5363 | 20950 | 0.0 | - | | 0.5376 | 21000 | 0.0 | - | | 0.5389 | 21050 | 0.0 | - | | 0.5402 | 21100 | 0.0 | - | | 0.5414 | 21150 | 0.0 | - | | 0.5427 | 21200 | 0.0 | - | | 0.5440 | 21250 | 0.0 | - | | 0.5453 | 21300 | 0.0 | - | | 0.5466 | 21350 | 0.0 | - | | 0.5478 | 21400 | 0.0 | - | | 0.5491 | 21450 | 0.0 | - | | 0.5504 | 21500 | 0.0 | - | | 0.5517 | 21550 | 0.0 | - | | 0.5530 | 21600 | 0.0 | - | | 0.5542 | 21650 | 0.0 | - | | 0.5555 | 21700 | 0.0 | - | | 0.5568 | 21750 | 0.0 | - | | 0.5581 | 21800 | 0.0 | - | | 0.5594 | 21850 | 0.0 | - | | 0.5606 | 21900 | 0.0 | - | | 0.5619 | 21950 | 0.0 | - | | 0.5632 | 22000 | 0.0 | - | | 0.5645 | 22050 | 0.0 | - | | 0.5658 | 22100 | 0.0 | - | | 0.5670 | 22150 | 0.0 | - | | 0.5683 | 22200 | 0.0 | - | | 0.5696 | 22250 | 0.0 | - | | 0.5709 | 22300 | 0.0 | - | | 0.5722 | 22350 | 0.0 | - | | 0.5734 | 22400 | 0.0 | - | | 0.5747 | 22450 | 0.0 | - | | 0.5760 | 22500 | 0.0 | - | | 0.5773 | 22550 | 0.0 | - | | 0.5786 | 22600 | 0.0 | - | | 0.5798 | 22650 | 0.0 | - | | 0.5811 | 22700 | 0.0 | - | | 0.5824 | 22750 | 0.0 | - | | 0.5837 | 22800 | 0.0 | - | | 0.5850 | 22850 | 0.0 | - | | 0.5862 | 22900 | 0.0 | - | | 0.5875 | 22950 | 0.0 | - | | 0.5888 | 23000 | 0.0 | - | | 0.5901 | 23050 | 0.0 | - | | 0.5914 | 23100 | 0.0 | - | | 0.5926 | 23150 | 0.0 | - | | 0.5939 | 23200 | 0.0 | - | | 0.5952 | 23250 | 0.0 | - | | 0.5965 | 23300 | 0.0 | - | | 0.5978 | 23350 | 0.0 | - | | 0.5990 | 23400 | 0.0 | - | | 0.6003 | 23450 | 0.0 | - | | 0.6016 | 23500 | 0.0 | - | | 0.6029 | 23550 | 0.0 | - | | 0.6042 | 23600 | 0.0 | - | | 0.6054 | 23650 | 0.0 | - | | 0.6067 | 23700 | 0.0 | - | | 0.6080 | 23750 | 0.0 | - | | 0.6093 | 23800 | 0.0 | - | | 0.6106 | 23850 | 0.0 | - | | 0.6118 | 23900 | 0.0 | - | | 0.6131 | 23950 | 0.0 | - | | 0.6144 | 24000 | 0.0 | - | | 0.6157 | 24050 | 0.0 | - | | 0.6170 | 24100 | 0.0 | - | | 0.6182 | 24150 | 0.0 | - | | 0.6195 | 24200 | 0.0 | - | | 0.6208 | 24250 | 0.0 | - | | 0.6221 | 24300 | 0.0 | - | | 0.6234 | 24350 | 0.0 | - | | 0.6246 | 24400 | 0.0 | - | | 0.6259 | 24450 | 0.0 | - | | 0.6272 | 24500 | 0.0 | - | | 0.6285 | 24550 | 0.0 | - | | 0.6298 | 24600 | 0.0 | - | | 0.6310 | 24650 | 0.0 | - | | 0.6323 | 24700 | 0.0 | - | | 0.6336 | 24750 | 0.0 | - | | 0.6349 | 24800 | 0.0 | - | | 0.6362 | 24850 | 0.0 | - | | 0.6374 | 24900 | 0.0 | - | | 0.6387 | 24950 | 0.0 | - | | 0.6400 | 25000 | 0.0 | - | | 0.6413 | 25050 | 0.0 | - | | 0.6426 | 25100 | 0.0 | - | | 0.6438 | 25150 | 0.0 | - | | 0.6451 | 25200 | 0.0 | - | | 0.6464 | 25250 | 0.0 | - | | 0.6477 | 25300 | 0.0 | - | | 0.6490 | 25350 | 0.0 | - | | 0.6502 | 25400 | 0.0 | - | | 0.6515 | 25450 | 0.0 | - | | 0.6528 | 25500 | 0.0 | - | | 0.6541 | 25550 | 0.0 | - | | 0.6554 | 25600 | 0.0 | - | | 0.6566 | 25650 | 0.0 | - | | 0.6579 | 25700 | 0.0 | - | | 0.6592 | 25750 | 0.0 | - | | 0.6605 | 25800 | 0.0 | - | | 0.6618 | 25850 | 0.0 | - | | 0.6630 | 25900 | 0.0 | - | | 0.6643 | 25950 | 0.0 | - | | 0.6656 | 26000 | 0.0 | - | | 0.6669 | 26050 | 0.0 | - | | 0.6682 | 26100 | 0.0 | - | | 0.6694 | 26150 | 0.0 | - | | 0.6707 | 26200 | 0.0 | - | | 0.6720 | 26250 | 0.0 | - | | 0.6733 | 26300 | 0.0 | - | | 0.6746 | 26350 | 0.0 | - | | 0.6758 | 26400 | 0.0 | - | | 0.6771 | 26450 | 0.0 | - | | 0.6784 | 26500 | 0.0 | - | | 0.6797 | 26550 | 0.0 | - | | 0.6810 | 26600 | 0.0 | - | | 0.6822 | 26650 | 0.0 | - | | 0.6835 | 26700 | 0.0 | - | | 0.6848 | 26750 | 0.0 | - | | 0.6861 | 26800 | 0.0 | - | | 0.6874 | 26850 | 0.0 | - | | 0.6886 | 26900 | 0.0 | - | | 0.6899 | 26950 | 0.0 | - | | 0.6912 | 27000 | 0.0 | - | | 0.6925 | 27050 | 0.0 | - | | 0.6938 | 27100 | 0.0 | - | | 0.6950 | 27150 | 0.0 | - | | 0.6963 | 27200 | 0.0 | - | | 0.6976 | 27250 | 0.0 | - | | 0.6989 | 27300 | 0.0 | - | | 0.7002 | 27350 | 0.0 | - | | 0.7014 | 27400 | 0.0 | - | | 0.7027 | 27450 | 0.0 | - | | 0.7040 | 27500 | 0.0 | - | | 0.7053 | 27550 | 0.0 | - | | 0.7066 | 27600 | 0.0 | - | | 0.7078 | 27650 | 0.0 | - | | 0.7091 | 27700 | 0.0 | - | | 0.7104 | 27750 | 0.0 | - | | 0.7117 | 27800 | 0.0 | - | | 0.7130 | 27850 | 0.0 | - | | 0.7142 | 27900 | 0.0 | - | | 0.7155 | 27950 | 0.0 | - | | 0.7168 | 28000 | 0.0 | - | | 0.7181 | 28050 | 0.0 | - | | 0.7194 | 28100 | 0.0 | - | | 0.7206 | 28150 | 0.0 | - | | 0.7219 | 28200 | 0.0 | - | | 0.7232 | 28250 | 0.0 | - | | 0.7245 | 28300 | 0.0 | - | | 0.7258 | 28350 | 0.0 | - | | 0.7270 | 28400 | 0.0 | - | | 0.7283 | 28450 | 0.0 | - | | 0.7296 | 28500 | 0.0 | - | | 0.7309 | 28550 | 0.0 | - | | 0.7322 | 28600 | 0.0 | - | | 0.7334 | 28650 | 0.0 | - | | 0.7347 | 28700 | 0.0 | - | | 0.7360 | 28750 | 0.0 | - | | 0.7373 | 28800 | 0.0 | - | | 0.7386 | 28850 | 0.0 | - | | 0.7398 | 28900 | 0.0 | - | | 0.7411 | 28950 | 0.0 | - | | 0.7424 | 29000 | 0.0 | - | | 0.7437 | 29050 | 0.0 | - | | 0.7450 | 29100 | 0.0 | - | | 0.7462 | 29150 | 0.0 | - | | 0.7475 | 29200 | 0.0 | - | | 0.7488 | 29250 | 0.0 | - | | 0.7501 | 29300 | 0.0 | - | | 0.7514 | 29350 | 0.0 | - | | 0.7526 | 29400 | 0.0 | - | | 0.7539 | 29450 | 0.0 | - | | 0.7552 | 29500 | 0.0 | - | | 0.7565 | 29550 | 0.0 | - | | 0.7578 | 29600 | 0.0 | - | | 0.7590 | 29650 | 0.0 | - | | 0.7603 | 29700 | 0.0 | - | | 0.7616 | 29750 | 0.0 | - | | 0.7629 | 29800 | 0.0 | - | | 0.7642 | 29850 | 0.0 | - | | 0.7654 | 29900 | 0.0 | - | | 0.7667 | 29950 | 0.0 | - | | 0.7680 | 30000 | 0.0 | - | | 0.7693 | 30050 | 0.0 | - | | 0.7706 | 30100 | 0.0 | - | | 0.7718 | 30150 | 0.0 | - | | 0.7731 | 30200 | 0.0 | - | | 0.7744 | 30250 | 0.0 | - | | 0.7757 | 30300 | 0.0 | - | | 0.7770 | 30350 | 0.0 | - | | 0.7782 | 30400 | 0.0 | - | | 0.7795 | 30450 | 0.0 | - | | 0.7808 | 30500 | 0.0 | - | | 0.7821 | 30550 | 0.0 | - | | 0.7833 | 30600 | 0.0 | - | | 0.7846 | 30650 | 0.0 | - | | 0.7859 | 30700 | 0.0 | - | | 0.7872 | 30750 | 0.0 | - | | 0.7885 | 30800 | 0.0 | - | | 0.7897 | 30850 | 0.0 | - | | 0.7910 | 30900 | 0.0 | - | | 0.7923 | 30950 | 0.0 | - | | 0.7936 | 31000 | 0.0 | - | | 0.7949 | 31050 | 0.0 | - | | 0.7961 | 31100 | 0.0 | - | | 0.7974 | 31150 | 0.0 | - | | 0.7987 | 31200 | 0.0 | - | | 0.8000 | 31250 | 0.0 | - | | 0.8013 | 31300 | 0.0 | - | | 0.8025 | 31350 | 0.0 | - | | 0.8038 | 31400 | 0.0 | - | | 0.8051 | 31450 | 0.0 | - | | 0.8064 | 31500 | 0.0 | - | | 0.8077 | 31550 | 0.0 | - | | 0.8089 | 31600 | 0.0 | - | | 0.8102 | 31650 | 0.0 | - | | 0.8115 | 31700 | 0.0 | - | | 0.8128 | 31750 | 0.0 | - | | 0.8141 | 31800 | 0.0 | - | | 0.8153 | 31850 | 0.0 | - | | 0.8166 | 31900 | 0.0 | - | | 0.8179 | 31950 | 0.0 | - | | 0.8192 | 32000 | 0.0 | - | | 0.8205 | 32050 | 0.0 | - | | 0.8217 | 32100 | 0.0 | - | | 0.8230 | 32150 | 0.0 | - | | 0.8243 | 32200 | 0.0 | - | | 0.8256 | 32250 | 0.0 | - | | 0.8269 | 32300 | 0.0 | - | | 0.8281 | 32350 | 0.0 | - | | 0.8294 | 32400 | 0.0 | - | | 0.8307 | 32450 | 0.0 | - | | 0.8320 | 32500 | 0.0 | - | | 0.8333 | 32550 | 0.0 | - | | 0.8345 | 32600 | 0.0 | - | | 0.8358 | 32650 | 0.0 | - | | 0.8371 | 32700 | 0.0 | - | | 0.8384 | 32750 | 0.0 | - | | 0.8397 | 32800 | 0.0 | - | | 0.8409 | 32850 | 0.0 | - | | 0.8422 | 32900 | 0.0 | - | | 0.8435 | 32950 | 0.0 | - | | 0.8448 | 33000 | 0.0 | - | | 0.8461 | 33050 | 0.0 | - | | 0.8473 | 33100 | 0.0 | - | | 0.8486 | 33150 | 0.0 | - | | 0.8499 | 33200 | 0.0 | - | | 0.8512 | 33250 | 0.0 | - | | 0.8525 | 33300 | 0.0 | - | | 0.8537 | 33350 | 0.0 | - | | 0.8550 | 33400 | 0.0 | - | | 0.8563 | 33450 | 0.0 | - | | 0.8576 | 33500 | 0.0 | - | | 0.8589 | 33550 | 0.0 | - | | 0.8601 | 33600 | 0.0 | - | | 0.8614 | 33650 | 0.0 | - | | 0.8627 | 33700 | 0.0 | - | | 0.8640 | 33750 | 0.0 | - | | 0.8653 | 33800 | 0.0 | - | | 0.8665 | 33850 | 0.0 | - | | 0.8678 | 33900 | 0.0 | - | | 0.8691 | 33950 | 0.0 | - | | 0.8704 | 34000 | 0.0 | - | | 0.8717 | 34050 | 0.0 | - | | 0.8729 | 34100 | 0.0 | - | | 0.8742 | 34150 | 0.0 | - | | 0.8755 | 34200 | 0.0 | - | | 0.8768 | 34250 | 0.0 | - | | 0.8781 | 34300 | 0.0 | - | | 0.8793 | 34350 | 0.0 | - | | 0.8806 | 34400 | 0.0 | - | | 0.8819 | 34450 | 0.0 | - | | 0.8832 | 34500 | 0.0 | - | | 0.8845 | 34550 | 0.0 | - | | 0.8857 | 34600 | 0.0 | - | | 0.8870 | 34650 | 0.0 | - | | 0.8883 | 34700 | 0.0 | - | | 0.8896 | 34750 | 0.0 | - | | 0.8909 | 34800 | 0.0 | - | | 0.8921 | 34850 | 0.0 | - | | 0.8934 | 34900 | 0.0 | - | | 0.8947 | 34950 | 0.0 | - | | 0.8960 | 35000 | 0.0 | - | | 0.8973 | 35050 | 0.0 | - | | 0.8985 | 35100 | 0.0 | - | | 0.8998 | 35150 | 0.0 | - | | 0.9011 | 35200 | 0.0 | - | | 0.9024 | 35250 | 0.0 | - | | 0.9037 | 35300 | 0.0 | - | | 0.9049 | 35350 | 0.0 | - | | 0.9062 | 35400 | 0.0 | - | | 0.9075 | 35450 | 0.0 | - | | 0.9088 | 35500 | 0.0 | - | | 0.9101 | 35550 | 0.0 | - | | 0.9113 | 35600 | 0.0 | - | | 0.9126 | 35650 | 0.0 | - | | 0.9139 | 35700 | 0.0 | - | | 0.9152 | 35750 | 0.0 | - | | 0.9165 | 35800 | 0.0 | - | | 0.9177 | 35850 | 0.0 | - | | 0.9190 | 35900 | 0.0 | - | | 0.9203 | 35950 | 0.0 | - | | 0.9216 | 36000 | 0.0 | - | | 0.9229 | 36050 | 0.0 | - | | 0.9241 | 36100 | 0.0 | - | | 0.9254 | 36150 | 0.0 | - | | 0.9267 | 36200 | 0.0 | - | | 0.9280 | 36250 | 0.0 | - | | 0.9293 | 36300 | 0.0 | - | | 0.9305 | 36350 | 0.0 | - | | 0.9318 | 36400 | 0.0 | - | | 0.9331 | 36450 | 0.0 | - | | 0.9344 | 36500 | 0.0 | - | | 0.9357 | 36550 | 0.0 | - | | 0.9369 | 36600 | 0.0 | - | | 0.9382 | 36650 | 0.0 | - | | 0.9395 | 36700 | 0.0 | - | | 0.9408 | 36750 | 0.0 | - | | 0.9421 | 36800 | 0.0 | - | | 0.9433 | 36850 | 0.0 | - | | 0.9446 | 36900 | 0.0 | - | | 0.9459 | 36950 | 0.0 | - | | 0.9472 | 37000 | 0.0 | - | | 0.9485 | 37050 | 0.0 | - | | 0.9497 | 37100 | 0.0 | - | | 0.9510 | 37150 | 0.0 | - | | 0.9523 | 37200 | 0.0 | - | | 0.9536 | 37250 | 0.0 | - | | 0.9549 | 37300 | 0.0 | - | | 0.9561 | 37350 | 0.0 | - | | 0.9574 | 37400 | 0.0 | - | | 0.9587 | 37450 | 0.0 | - | | 0.9600 | 37500 | 0.0 | - | | 0.9613 | 37550 | 0.0 | - | | 0.9625 | 37600 | 0.0 | - | | 0.9638 | 37650 | 0.0 | - | | 0.9651 | 37700 | 0.0 | - | | 0.9664 | 37750 | 0.0 | - | | 0.9677 | 37800 | 0.0 | - | | 0.9689 | 37850 | 0.0 | - | | 0.9702 | 37900 | 0.0 | - | | 0.9715 | 37950 | 0.0 | - | | 0.9728 | 38000 | 0.0 | - | | 0.9741 | 38050 | 0.0 | - | | 0.9753 | 38100 | 0.0 | - | | 0.9766 | 38150 | 0.0 | - | | 0.9779 | 38200 | 0.0 | - | | 0.9792 | 38250 | 0.0 | - | | 0.9805 | 38300 | 0.0 | - | | 0.9817 | 38350 | 0.0 | - | | 0.9830 | 38400 | 0.0 | - | | 0.9843 | 38450 | 0.0 | - | | 0.9856 | 38500 | 0.0 | - | | 0.9869 | 38550 | 0.0 | - | | 0.9881 | 38600 | 0.0 | - | | 0.9894 | 38650 | 0.0 | - | | 0.9907 | 38700 | 0.0 | - | | 0.9920 | 38750 | 0.0 | - | | 0.9933 | 38800 | 0.0 | - | | 0.9945 | 38850 | 0.0 | - | | 0.9958 | 38900 | 0.0 | - | | 0.9971 | 38950 | 0.0 | - | | 0.9984 | 39000 | 0.0 | - | | 0.9997 | 39050 | 0.0 | - | | **1.0** | **39063** | **-** | **0.4016** | * The bold row denotes the saved checkpoint. ### Framework Versions - Python: 3.10.13 - SetFit: 1.0.3 - Sentence Transformers: 2.2.2 - Transformers: 4.36.2 - PyTorch: 2.1.2+cu121 - Datasets: 2.16.1 - Tokenizers: 0.15.0 ## Citation ### BibTeX ```bibtex @article{https://doi.org/10.48550/arxiv.2209.11055, doi = {10.48550/ARXIV.2209.11055}, url = {https://arxiv.org/abs/2209.11055}, author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren}, keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences}, title = {Efficient Few-Shot Learning Without Prompts}, publisher = {arXiv}, year = {2022}, copyright = {Creative Commons Attribution 4.0 International} } ```