fm4bio-ning commited on
Commit
021cbab
1 Parent(s): b460f23

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +8 -4
README.md CHANGED
@@ -10,7 +10,7 @@ By leveraging MoE layers, AIDO.Protein efficiently scales to 16 billion paramete
10
 
11
  ## Model Architecture Details
12
  ADIO.Protein is a transformer encoder-only architecture with the dense MLP layer in each transformer block replaced by a sparse MoE layer. It uses single amino acid tokenization and is optimized using a masked languange modeling (MLM) training objective. For each token, 2 experts will be selectively activated by the top-2 rounting mechiansim.
13
- <center><img src="Proteion-MOE architecture.png" alt="An Overview of AIDO.Protein" style="width:70%; height:auto;" /></center>
14
  More architecture details are shown below:
15
 
16
  |Model Arch Component | Value |
@@ -49,11 +49,15 @@ We encode protein sequence with single amino acid resolution with 44 vocabularie
49
  We assess the advantages of pretraining AIDO.Protein 16B through experiments across more than 300 tasks from two important protein benchmarks, xTrimoPGLM benchmark and ProteinGym DMS benchmark, encompassing residue-level, sequence-level, and protein-protein interaction (PPI) level tasks. We further adapted our model for structure-conditioned protein sequence generation tasks
50
 
51
  ## Results
52
- - xTrimoPGLM Benchmark
 
53
 
54
- - ProteinGym DMS Benchmark
 
 
 
 
55
 
56
- - Inverse Folding Generation
57
 
58
  ## How to Use
59
  ### Build any downstream models from this backbone
 
10
 
11
  ## Model Architecture Details
12
  ADIO.Protein is a transformer encoder-only architecture with the dense MLP layer in each transformer block replaced by a sparse MoE layer. It uses single amino acid tokenization and is optimized using a masked languange modeling (MLM) training objective. For each token, 2 experts will be selectively activated by the top-2 rounting mechiansim.
13
+ <center><img src="proteinmoe_architecture.png" alt="An Overview of AIDO.Protein" style="width:70%; height:auto;" /></center>
14
  More architecture details are shown below:
15
 
16
  |Model Arch Component | Value |
 
49
  We assess the advantages of pretraining AIDO.Protein 16B through experiments across more than 300 tasks from two important protein benchmarks, xTrimoPGLM benchmark and ProteinGym DMS benchmark, encompassing residue-level, sequence-level, and protein-protein interaction (PPI) level tasks. We further adapted our model for structure-conditioned protein sequence generation tasks
50
 
51
  ## Results
52
+ ### xTrimoPGLM Benchmark
53
+ <center><img src="xtrimo_results.png" alt="An Overview of AIDO.Protein" style="width:70%; height:auto;" /></center>
54
 
55
+ ### ProteinGym DMS Benchmark
56
+ <center><img src="dms_results.png" alt="An Overview of AIDO.Protein" style="width:70%; height:auto;" /></center>
57
+
58
+ ### Inverse Folding Generation
59
+ <center><img src="inverse_folding.png" alt="An Overview of AIDO.Protein" style="width:70%; height:auto;" /></center>
60
 
 
61
 
62
  ## How to Use
63
  ### Build any downstream models from this backbone