Commit
·
7221211
1
Parent(s):
db25331
Update README.md
Browse files
README.md
CHANGED
@@ -5,13 +5,16 @@ language:
|
|
5 |
tags:
|
6 |
- cybersecurity
|
7 |
---
|
|
|
8 |
This model represents an improved version of the [SecureBERT](https://huggingface.co/ehsanaghaei/SecureBERT) model, trained on a corpus eight times larger than its predecessor, leveraging the computational power of 8xA100 GPUs. This version, known as SecureBERT+, brings forth an average improvment of 9% in the performance of the Masked Language Model (MLM) task. This advancement signifies a substantial stride towards achieving heightened proficiency in language understanding and representation learning within the cybersecurity domain.
|
9 |
|
10 |
|
11 |
SecureBERT is a domain-specific language model based on RoBERTa which is trained on a huge amount of cybersecurity data and fine-tuned/tweaked to understand/represent cybersecurity textual data.
|
12 |
|
|
|
|
|
13 |
|
14 |
-
|
15 |
|
16 |
[SecureGPT](https://huggingface.co/ehsanaghaei/SecureGPT)
|
17 |
|
|
|
5 |
tags:
|
6 |
- cybersecurity
|
7 |
---
|
8 |
+
# SecureBERT+
|
9 |
This model represents an improved version of the [SecureBERT](https://huggingface.co/ehsanaghaei/SecureBERT) model, trained on a corpus eight times larger than its predecessor, leveraging the computational power of 8xA100 GPUs. This version, known as SecureBERT+, brings forth an average improvment of 9% in the performance of the Masked Language Model (MLM) task. This advancement signifies a substantial stride towards achieving heightened proficiency in language understanding and representation learning within the cybersecurity domain.
|
10 |
|
11 |
|
12 |
SecureBERT is a domain-specific language model based on RoBERTa which is trained on a huge amount of cybersecurity data and fine-tuned/tweaked to understand/represent cybersecurity textual data.
|
13 |
|
14 |
+
## Dataset
|
15 |
+

|
16 |
|
17 |
+
Other model variants:
|
18 |
|
19 |
[SecureGPT](https://huggingface.co/ehsanaghaei/SecureGPT)
|
20 |
|