Update README.md
Browse files
README.md
CHANGED
|
@@ -5,6 +5,11 @@ tags:
|
|
| 5 |
- mpnet
|
| 6 |
- classification
|
| 7 |
- fine-tuned
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 8 |
---
|
| 9 |
|
| 10 |
# AttackGroup-MPNET - Model Card for MPNet Cybersecurity Classifier
|
|
@@ -190,14 +195,38 @@ To be anounced...
|
|
| 190 |
|
| 191 |
## Evaluation Results
|
| 192 |
|
| 193 |
-
| Model
|
| 194 |
-
|
| 195 |
-
|
|
| 196 |
-
|
|
| 197 |
-
|
|
| 198 |
-
|
|
| 199 |
-
|
|
| 200 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 201 |
|
| 202 |
|
| 203 |
### Single Prediction Example
|
|
|
|
| 5 |
- mpnet
|
| 6 |
- classification
|
| 7 |
- fine-tuned
|
| 8 |
+
license: creativeml-openrail-m
|
| 9 |
+
language:
|
| 10 |
+
- en
|
| 11 |
+
base_model:
|
| 12 |
+
- sentence-transformers/all-mpnet-base-v2
|
| 13 |
---
|
| 14 |
|
| 15 |
# AttackGroup-MPNET - Model Card for MPNet Cybersecurity Classifier
|
|
|
|
| 195 |
|
| 196 |
## Evaluation Results
|
| 197 |
|
| 198 |
+
| Model | Accuracy | F1 Macro | F1 Weighted | Embedding Variability |
|
| 199 |
+
|-----------------------|----------|----------|-------------|-----------------------|
|
| 200 |
+
| **AttackGroup-MPNET** | **0.85** | **0.759**| **0.847** | 0.234 |
|
| 201 |
+
| GTE Large | 0.66 | 0.571 | 0.667 | 0.266 |
|
| 202 |
+
| E5 Large v2 | 0.64 | 0.541 | 0.650 | 0.355 |
|
| 203 |
+
| Original MPNet | 0.63 | 0.534 | 0.619 | 0.092 |
|
| 204 |
+
| BGE Large | 0.53 | 0.418 | 0.519 | 0.366 |
|
| 205 |
+
| SupSimCSE | 0.50 | 0.373 | 0.479 | 0.227 |
|
| 206 |
+
| MLM Fine-tuned MPNet | 0.44 | 0.272 | 0.411 | 0.125 |
|
| 207 |
+
| SecBERT | 0.41 | 0.315 | 0.410 | 0.591 |
|
| 208 |
+
| SecureBERT_Plus | 0.36 | 0.252 | 0.349 | 0.267 |
|
| 209 |
+
| CySecBERT | 0.34 | 0.235 | 0.323 | 0.229 |
|
| 210 |
+
| ATTACK-BERT | 0.33 | 0.240 | 0.316 | 0.096 |
|
| 211 |
+
| Secure_BERT | 0.00 | 0.000 | 0.000 | 0.007 |
|
| 212 |
+
| CyBERT | 0.00 | 0.000 | 0.000 | 0.015 |
|
| 213 |
+
|
| 214 |
+
|
| 215 |
+
| Model | Similarity Search Recall@5 | Few-shot Accuracy | In-dist Similarity | OOD Similarity | Robustness Similarity |
|
| 216 |
+
|----------------------|----------------------------|-------------------|--------------------|----------------|-----------------------|
|
| 217 |
+
| **AttackGroup-MPNET**| **0.934** | **0.857** | 0.235 | 0.017 | 0.948 |
|
| 218 |
+
| Original MPNet | 0.786 | 0.643 | 0.217 | -0.004 | 0.941 |
|
| 219 |
+
| E5 Large v2 | 0.778 | 0.679 | 0.727 | 0.013 | 0.977 |
|
| 220 |
+
| GTE Large | 0.746 | 0.786 | 0.845 | 0.002 | 0.984 |
|
| 221 |
+
| BGE Large | 0.632 | 0.750 | 0.533 | -0.006 | 0.970 |
|
| 222 |
+
| SupSimCSE | 0.616 | 0.571 | 0.683 | -0.015 | 0.978 |
|
| 223 |
+
| SecBERT | 0.468 | 0.429 | 0.586 | -0.001 | 0.970 |
|
| 224 |
+
| CyBERT | 0.452 | 0.250 | 1.000 | -0.001 | 1.000 |
|
| 225 |
+
| ATTACK-BERT | 0.362 | 0.571 | 0.157 | -0.005 | 0.950 |
|
| 226 |
+
| CySecBERT | 0.424 | 0.500 | 0.734 | -0.015 | 0.954 |
|
| 227 |
+
| Secure_BERT | 0.424 | 0.250 | 0.990 | 0.050 | 0.998 |
|
| 228 |
+
| SecureBERT_Plus | 0.406 | 0.464 | 0.981 | 0.040 | 0.998 |
|
| 229 |
+
|
| 230 |
|
| 231 |
|
| 232 |
### Single Prediction Example
|