metadata

license: cc-by-nc-sa-4.0
tags:
  - chemistry
  - drug-design
  - synthesis-accessibility
  - cheminformatics
  - drug-discovery
  - selfies
  - drugs
  - molecules
  - compounds
  - ranger21
  - madgrad

Model Card for ChemFIE-SA (Synthesis Accessibility)

This model is a BERT-like sequence classifier for 221 human protein drug targets, fine-tuned from gbyuvd/chemselfies-base-bertmlm on a dataset derived from ChemBL34 (Zdrazil et al. 2023). It predicts using chemical structures represented as SELFIES (Self-Referencing Embedded Strings).

Disclaimer: For Academic Purposes Only

The information and model provided is for academic purposes only. It is intended for educational and research use, and should not be used for any commercial or legal purposes. The author do not guarantee the accuracy, completeness, or reliability of the information.

Model Details

Model Description

Model Type: Transformer (BertForSequenceClassification)
Base model: gbyuvd/chemselfies-base-bertmlm
Maximum Sequence Length: 512 tokens
Number of Labels: 2 classes (0 ES: easy synthesis; 1 HS: hard to synthesize)
Training Dataset: SELFIES with labels derived from DeepSA
Language: SELFIES
License: CC-BY-NC-SA 4.0

Uses

If you have Canonical SMILES instead of SELFIES, you can convert it first into a format readable by the model's tokenizer (using whitespace)

import selfies as sf

def smiles_to_selfies_sentence(smiles):
    try:
        selfies = sf.encoder(smiles)  # Encode SMILES into SELFIES
        selfies_tokens = list(sf.split_selfies(selfies))
        
        # Join dots with the nearest next tokens
        joined_tokens = []
        i = 0
        while i < len(selfies_tokens):
            if selfies_tokens[i] == '.' and i + 1 < len(selfies_tokens):
                joined_tokens.append(f".{selfies_tokens[i+1]}")
                i += 2
            else:
                joined_tokens.append(selfies_tokens[i])
                i += 1
        
        selfies_sentence = ' '.join(joined_tokens)
        return selfies_sentence
    except sf.EncoderError as e:
        print(f"Encoder Error: {e}")
        return None

# Example usage:
in_smi = "C1CCC2=CN3C=CC4=C5C=CC=CC5=NC4=C3C=C2C1" # Sempervirine (CID168919)
selfies_sentence = smiles_to_selfies_sentence(in_smi)
print(selfies_sentence)

"""
[C] [C] [C] [C] [=C] [N] [C] [=C] [C] [=C] [C] [=C] [C] [=C] [C] [Ring1] [=Branch1] [=N] [C] [Ring1] [=Branch2] [=C] [Ring1] [=N] [C] [=C] [Ring1] [P] [C] [Ring2] [Ring1] [Branch1]

"""

Direct Use using Classifier Pipeline

You can also use pipeline:

from transformers import pipeline

classifier = pipeline("text-classification", model="gbyuvd/synthaccess-chemselfies")
classifier("[C] [C] [C] [C] [=C] [N] [C] [=C] [C] [=C] [C] [=C] [C] [=C] [C] [Ring1] [=Branch1] [=N] [C] [Ring1] [=Branch2] [=C] [Ring1] [=N] [C] [=C] [Ring1] [P] [C] [Ring2] [Ring1] [Branch1]") #Sempervirine (CID168919)
#

Out-of-Scope Use

[More Information Needed]

Bias, Risks, and Limitations

[More Information Needed]

Recommendations

Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.

Training Details

Training Data

Data Sources

Data Preparation

[More Information Needed]

Training Procedure

Training Hyperparameters

Training regime: [More Information Needed]

Evaluation

Testing Data, Factors & Metrics

Testing Data

[More Information Needed]

Factors

[More Information Needed]

Metrics

[More Information Needed]

Results

[More Information Needed]

Summary

Model Examination

You can visualize its attention heads using BertViz and attribution weights using Captum - as done in the base model in Interpretability section.

Model Architecture and Objective

[More Information Needed]

Compute Infrastructure

Hardware

Platform: Paperspace's Gradients
Compute: Free-P5000 (16 GB GPU, 30 GB RAM, 8 vCPU)

Software

Python: 3.9.13
Transformers: 4.42.4
PyTorch: 2.3.1+cu121
Accelerate: 0.32.0
Datasets: 2.20.0
Tokenizers: 0.19.1
Ranger21: 0.0.1
Selfies: 2.1.2
RDKit: 2024.3.3

Citation

If you find this project useful in your research and wish to cite it, please use the following BibTex entry:

@software{chemfie_basebertmlm,
  author = {GP Bayu},
  title = {{ChemFIE Base}: Pretraining A Lightweight BERT-like model on Molecular SELFIES},
  url = {https://huggingface.co/gbyuvd/chemselfies-base-bertmlm},
  version = {1.0},
  year = {2024},
}

References

DeepSA

@article{Wang2023DeepSA,
  title={DeepSA: a deep-learning driven predictor of compound synthesis accessibility},
  author={Wang, Shihang and Wang, Lin and Li, Fenglei and Bai, Fang},
  journal={Journal of Cheminformatics},
  volume={15},
  pages={103},
  year={2023},
  month={Nov},
  publisher={BioMed Central},
  doi={10.1186/s13321-023-00771-3},
}

Contact & Support My Work

G Bayu ([email protected])

This project has been quiet a journey for me, I’ve dedicated hours on this and I would like to improve myself, this model, and future projects. However, financial and computational constraints can be challenging.

If you find my work valuable and would like to support my journey, please consider supporting me here. Your support will help me cover costs for computational resources, data acquisition, and further development of this project. Any amount, big or small, is greatly appreciated and will enable me to continue learning and explore more.

Thank you for checking out this model, I am more than happy to receive any feedback, so that I can improve myself and the future model/projects I will be working on.