Update README.md
Browse files
README.md
CHANGED
@@ -44,6 +44,7 @@ python bin/PhaStyle.py \
|
|
44 |
|
45 |
```
|
46 |
|
|
|
47 |
### Datasets Used:
|
48 |
|
49 |
- **BACPHLIP (without E. coli)**: 1,868 training sequences and 246 validation sequences.
|
@@ -53,18 +54,55 @@ python bin/PhaStyle.py \
|
|
53 |
Each dataset was processed using **512bp segment lengths** to simulate fragmented metagenomic assemblies.
|
54 |
|
55 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
56 |
|
57 |
-
|
58 |
|
59 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
60 |
|
61 |
-
|
62 |
-
- **Balanced Accuracy**: 0.94 (on 1022bp fragments from the *Escherichia* dataset)
|
63 |
-
- **MCC (Matthews Correlation Coefficient)**: 0.91
|
64 |
-
- **Sensitivity**: 0.97
|
65 |
-
- **Specificity**: 0.91
|
66 |
|
|
|
67 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
68 |
|
69 |
## Limitations
|
70 |
|
|
|
44 |
|
45 |
```
|
46 |
|
47 |
+
|
48 |
### Datasets Used:
|
49 |
|
50 |
- **BACPHLIP (without E. coli)**: 1,868 training sequences and 246 validation sequences.
|
|
|
54 |
Each dataset was processed using **512bp segment lengths** to simulate fragmented metagenomic assemblies.
|
55 |
|
56 |
---
|
57 |
+
## Performance Results
|
58 |
+
|
59 |
+
The performance of ProkBERT PhaStyle was evaluated on various datasets, including *Escherichia* and EXTREMOPHILE phages, using segment lengths of 512bp and 1022bp. The results are summarized below:
|
60 |
+
|
61 |
+
### Performance on *Escherichia* Dataset (512bp and 1022bp segments)
|
62 |
+
|
63 |
+
| Method | Balanced Accuracy | MCC | Sensitivity | Specificity |
|
64 |
+
|--------------------------|-------------------|-------|-------------|-------------|
|
65 |
+
| **ProkBERT-mini (512bp)** | 0.91 | 0.83 | 0.94 | 0.89 |
|
66 |
+
| ProkBERT-mini-long (512bp)| 0.90 | 0.82 | 0.96 | 0.85 |
|
67 |
+
| ProkBERT-mini-c (512bp) | 0.89 | 0.80 | 0.95 | 0.84 |
|
68 |
+
| DNABERT-2-117M (512bp) | 0.84 | 0.72 | 0.95 | 0.74 |
|
69 |
+
| Nuc. Trans.-50m (512bp) | 0.85 | 0.72 | 0.92 | 0.78 |
|
70 |
+
| **ProkBERT-mini (1022bp)**| **0.94** | **0.88** | **0.97** | **0.91** |
|
71 |
+
| ProkBERT-mini-long (1022bp)| 0.94 | 0.89 | 0.97 | 0.91 |
|
72 |
|
73 |
+
### Performance on EXTREMOPHILE Dataset (512bp and 1022bp segments)
|
74 |
|
75 |
+
| Method | Balanced Accuracy | MCC | Sensitivity | Specificity |
|
76 |
+
|--------------------------|-------------------|-------|-------------|-------------|
|
77 |
+
| **ProkBERT-mini (512bp)** | 0.93 | 0.83 | 0.99 | 0.87 |
|
78 |
+
| ProkBERT-mini-long (512bp)| 0.93 | 0.82 | **1.00** | 0.86 |
|
79 |
+
| ProkBERT-mini-c (512bp) | 0.92 | 0.80 | 0.99 | 0.84 |
|
80 |
+
| DNABERT-2-117M (512bp) | 0.89 | 0.74 | 0.99 | 0.79 |
|
81 |
+
| **ProkBERT-mini (1022bp)**| **0.96** | **0.91** | **1.00** | **0.93** |
|
82 |
+
| ProkBERT-mini-long (1022bp)| 0.96 | 0.90 | 1.00 | 0.92 |
|
83 |
|
84 |
+
These tables highlight the high accuracy, MCC, and generalization capability of ProkBERT models, particularly on challenging datasets like *Escherichia* and extremophile phages. The ProkBERT-mini and ProkBERT-mini-long models consistently performed well on both datasets.
|
|
|
|
|
|
|
|
|
85 |
|
86 |
+
For more detailed results, including additional metrics, please refer to the original research paper.
|
87 |
---
|
88 |
+
## Inference Speed and Running Times
|
89 |
+
|
90 |
+
The computational performance of ProkBERT PhaStyle was evaluated using 1,000 randomly selected sequences from the BACPHLIP dataset. The evaluation was performed on a consistent hardware setup with NVIDIA Tesla A100 GPUs. The execution times and inference speeds of various models are summarized below:
|
91 |
+
|
92 |
+
### Execution Times (in seconds)
|
93 |
+
|
94 |
+
| Model | Execution Time (seconds) | Inference Speed (MB/sec) |
|
95 |
+
|--------------------------|--------------------------|--------------------------|
|
96 |
+
| **ProkBERT-mini-long** | **132** | **0.52** |
|
97 |
+
| ProkBERT-mini | 141 | 0.49 |
|
98 |
+
| ProkBERT-mini-c | 146 | 0.47 |
|
99 |
+
| DNABERT-2-117M | 248 | 0.25 |
|
100 |
+
| Nucleotide Transformer-50m| 342 | 0.18 |
|
101 |
+
| Nucleotide Transformer-500m| 502 | 0.12 |
|
102 |
+
| DeePhage | 159 | 0.43 |
|
103 |
+
| PhaTYP | 2,718 | 0.03 |
|
104 |
+
| BACPHLIP | 7,125 | 0.01 |
|
105 |
+
|
106 |
|
107 |
## Limitations
|
108 |
|