Update README.md

README.md

@@ -1,5 +1,73 @@
 ---
-license: apache-2.0
+license: mit
+datasets:
+- sagawa/pubchem-10m-canonicalized
+metrics:
+- accuracy
+model-index:
+- name: PubChem-10m-t5
+  results:
+  - task:
+      name: Masked Language Modeling
+      type: fill-mask
+    dataset:
+      name: sagawa/pubchem-10m-canonicalized
+      type: sagawa/pubchem-10m-canonicalized
+    metrics:
+    - name: Accuracy
+      type: accuracy
+      value: 0.9259435534477234
 ---
+
 # PubChem-10m-t5
-We trained T5 on SMILES from PubChem using the task of masked-language modeling (MLM), and its tokenizer is also trained on PubChem data. This model can be used for the prediction of molecules' properties, reactions, or interactions with proteins by changing the way of finetuning.
+
+This model is a fine-tuned version of [google/t5-v1_1-base](https://huggingface.co/microsoft/deberta-base) on the sagawa/pubchem-10m-canonicalized dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.2121
+- Accuracy: 0.9259
+
+
+## Model description
+
+We trained t5 on SMILES from PubChem using the task of masked-language modeling (MLM). Its tokenizer is also trained on PubChem.
+
+
+## Intended uses & limitations
+
+This model can be used for the prediction of molecules' properties, reactions, or interactions with proteins by changing the way of finetuning.
+
+## Training and evaluation data
+
+We downloaded [PubChem data](https://drive.google.com/file/d/1ygYs8dy1-vxD1Vx6Ux7ftrXwZctFjpV3/view) and canonicalized them using RDKit. Then, we dropped duplicates. The total number of data is 9999960, and they were randomly split into train:validation=10:1.
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 5e-03
+- train_batch_size: 30
+- eval_batch_size: 32
+- seed: 42
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- num_epochs: 30.0
+
+### Training results
+
+| Training Loss | Step   | Accuracy | Validation Loss |
+|:-------------:|:------:|:--------:|:---------------:|
+| 0.3866        | 25000  | 0.8830   | 0.3631          |
+| 0.3352        | 50000  | 0.8996   | 0.3049          |
+| 0.2834        | 75000  | 0.9057   | 0.2825          |
+| 0.2685        | 100000 | 0.9099   | 0.2675          |
+| 0.2591        | 125000 | 0.9124   | 0.2587          |
+| 0.2620        | 150000 | 0.9144   | 0.2512          |
+| 0.2806        | 175000 | 0.9161   | 0.2454          |
+| 0.2468        | 200000 | 0.9179   | 0.2396          |
+| 0.2669        | 225000 | 0.9194   | 0.2343          |
+| 0.2611        | 250000 | 0.9210   | 0.2283          |
+| 0.2346        | 275000 | 0.9226   | 0.2230          |
+| 0.1972        | 300000 | 0.9238   | 0.2191          |
+| 0.2344        | 325000 | 0.9250   | 0.2152          |
+| 0.2164        | 350000 | 0.9259   | 0.2121          |