pszemraj commited on
Commit
d0aecc8
·
verified ·
1 Parent(s): 501873a

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +9 -0
README.md CHANGED
@@ -750,6 +750,15 @@ Thus far, all completed in fp32 (_using nvidia tf32 dtype behind the scenes when
750
  | bert-base-uncased | 110M | 52.1 | 93.5 | 88.9 | 85.8 | 71.2 | 84.0 | 90.5 | 66.4 | 79.05 |
751
  | roberta-base | 125M | 64.0 | 95.0 | 90.0 | 91.0 | 92.0 | 88.0 | 93.0 | 79.0 | 86.5 |
752
 
 
 
 
 
 
 
 
 
 
753
  ### Observations:
754
 
755
  1. **Performance Variation Across Models and Tasks**: The data highlights significant performance variability both across and within models for different GLUE tasks. This variability underscores the complexity of natural language understanding tasks and the need for models to be versatile in handling different types of linguistic challenges.
 
750
  | bert-base-uncased | 110M | 52.1 | 93.5 | 88.9 | 85.8 | 71.2 | 84.0 | 90.5 | 66.4 | 79.05 |
751
  | roberta-base | 125M | 64.0 | 95.0 | 90.0 | 91.0 | 92.0 | 88.0 | 93.0 | 79.0 | 86.5 |
752
 
753
+ and some comparisons to recent BERT models taken from [nomic's blog post]():
754
+
755
+ | Model | Size | CoLA | SST2 | MRPC | STSB | QQP | MNLI | QNLI | RTE | Avg |
756
+ |---------------|-------|-------|------|------|------|------|------|------|------|-------|
757
+ | NomicBERT | 137M | 50.00 | 93.00| 88.00| 90.00| 92.00| 86.00| 92.00| 82.00| 84.00 |
758
+ | RobertaBase | 125M | 64.00 | 95.00| 90.00| 91.00| 92.00| 88.00| 93.00| 79.00| 86.00 |
759
+ | JinaBERTBase | 137M | 51.00 | 95.00| 88.00| 90.00| 81.00| 86.00| 92.00| 79.00| 83.00 |
760
+ | MosaicBERT | 137M (??) | 59.00 | 94.00| 89.00| 90.00| 92.00| 86.00| 91.00| 83.00| 85.00 |
761
+
762
  ### Observations:
763
 
764
  1. **Performance Variation Across Models and Tasks**: The data highlights significant performance variability both across and within models for different GLUE tasks. This variability underscores the complexity of natural language understanding tasks and the need for models to be versatile in handling different types of linguistic challenges.