README.md

metadata

tags:
  - generated_from_trainer
model-index:
  - name: SMILES_BERT
    results: []

SMILES_BERT

A BERT model trained on a list of 50,000 SMILES for MLM

Model description

This model is a BERT model that was trained on a list of 50k SMILES. The SMILES were sourced from BindingDB and the compounds bind to certain proteins with some affinity. The purpose of this model was to provide a model which can then be fine-tuned for other tasks in which SMILES data can be useful.

Intended uses & limitations

This model was trained in order to provide a model which can then be fine-tuned for other tasks in which SMILES data can be useful such as predicting physical properties, chemical activity, or biological activity.

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 64
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 3

Training results

Training Loss: 0.9446000

Further evaluation is needed

Framework versions

• Transformers 4.37.0.dev0
• Pytorch 2.1.0+cu121
• Tokenizers 0.15.0