|
--- |
|
license: cc-by-nc-sa-4.0 |
|
tags: |
|
- chemistry |
|
- drug-design |
|
- synthesis-accessibility |
|
- cheminformatics |
|
- drug-discovery |
|
- selfies |
|
- drugs |
|
- molecules |
|
- compounds |
|
- ranger21 |
|
- madgrad |
|
--- |
|
|
|
# Model Card for ChemFIE-SA (Synthesis Accessibility) |
|
|
|
This model is a BERT-like sequence classifier for 221 human protein drug targets, fine-tuned from [gbyuvd/chemselfies-base-bertmlm](https://huggingface.co/gbyuvd/chemselfies-base-bertmlm) on a dataset derived from ChemBL34 (Zdrazil et al. 2023). It predicts using chemical structures represented as SELFIES (Self-Referencing Embedded Strings). |
|
|
|
|
|
### Disclaimer: For Academic Purposes Only |
|
The information and model provided is for academic purposes only. It is intended for educational and research use, and should not be used for any commercial or legal purposes. The author do not guarantee the accuracy, completeness, or reliability of the information. |
|
|
|
|
|
[](https://ko-fi.com/O4O710GFBZ) |
|
|
|
## Model Details |
|
|
|
### Model Description |
|
|
|
- **Model Type:** Transformer (BertForSequenceClassification) |
|
- **Base model:** [gbyuvd/chemselfies-base-bertmlm](https://huggingface.co/gbyuvd/chemselfies-base-bertmlm) |
|
- **Maximum Sequence Length:** 512 tokens |
|
- **Number of Labels:** 2 classes (0 ES: easy synthesis; 1 HS: hard to synthesize) |
|
- **Training Dataset:** SELFIES with labels derived from DeepSA |
|
- **Language:** SELFIES |
|
- **License:** CC-BY-NC-SA 4.0 |
|
|
|
## Uses |
|
|
|
If you have Canonical SMILES instead of SELFIES, you can convert it first into a format readable by the model's tokenizer (using whitespace) |
|
|
|
```python |
|
import selfies as sf |
|
|
|
def smiles_to_selfies_sentence(smiles): |
|
try: |
|
selfies = sf.encoder(smiles) # Encode SMILES into SELFIES |
|
selfies_tokens = list(sf.split_selfies(selfies)) |
|
|
|
# Join dots with the nearest next tokens |
|
joined_tokens = [] |
|
i = 0 |
|
while i < len(selfies_tokens): |
|
if selfies_tokens[i] == '.' and i + 1 < len(selfies_tokens): |
|
joined_tokens.append(f".{selfies_tokens[i+1]}") |
|
i += 2 |
|
else: |
|
joined_tokens.append(selfies_tokens[i]) |
|
i += 1 |
|
|
|
selfies_sentence = ' '.join(joined_tokens) |
|
return selfies_sentence |
|
except sf.EncoderError as e: |
|
print(f"Encoder Error: {e}") |
|
return None |
|
|
|
# Example usage: |
|
in_smi = "C1CCC2=CN3C=CC4=C5C=CC=CC5=NC4=C3C=C2C1" # Sempervirine (CID168919) |
|
selfies_sentence = smiles_to_selfies_sentence(in_smi) |
|
print(selfies_sentence) |
|
|
|
""" |
|
[C] [C] [C] [C] [=C] [N] [C] [=C] [C] [=C] [C] [=C] [C] [=C] [C] [Ring1] [=Branch1] [=N] [C] [Ring1] [=Branch2] [=C] [Ring1] [=N] [C] [=C] [Ring1] [P] [C] [Ring2] [Ring1] [Branch1] |
|
|
|
""" |
|
|
|
``` |
|
|
|
### Direct Use using Classifier Pipeline |
|
|
|
You can also use pipeline: |
|
|
|
```python |
|
from transformers import pipeline |
|
|
|
classifier = pipeline("text-classification", model="gbyuvd/synthaccess-chemselfies") |
|
classifier("[C] [C] [C] [C] [=C] [N] [C] [=C] [C] [=C] [C] [=C] [C] [=C] [C] [Ring1] [=Branch1] [=N] [C] [Ring1] [=Branch2] [=C] [Ring1] [=N] [C] [=C] [Ring1] [P] [C] [Ring2] [Ring1] [Branch1]") #Sempervirine (CID168919) |
|
# |
|
|
|
``` |
|
|
|
### Out-of-Scope Use |
|
|
|
<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. --> |
|
|
|
[More Information Needed] |
|
|
|
## Bias, Risks, and Limitations |
|
|
|
<!-- This section is meant to convey both technical and sociotechnical limitations. --> |
|
|
|
[More Information Needed] |
|
|
|
### Recommendations |
|
|
|
<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. --> |
|
|
|
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations. |
|
|
|
|
|
## Training Details |
|
|
|
### Training Data |
|
##### Data Sources |
|
##### Data Preparation |
|
|
|
<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. --> |
|
|
|
[More Information Needed] |
|
|
|
### Training Procedure |
|
|
|
#### Training Hyperparameters |
|
|
|
- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision --> |
|
|
|
|
|
## Evaluation |
|
|
|
<!-- This section describes the evaluation protocols and provides the results. --> |
|
|
|
### Testing Data, Factors & Metrics |
|
|
|
#### Testing Data |
|
|
|
<!-- This should link to a Dataset Card if possible. --> |
|
|
|
[More Information Needed] |
|
|
|
#### Factors |
|
|
|
<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. --> |
|
|
|
[More Information Needed] |
|
|
|
#### Metrics |
|
|
|
<!-- These are the evaluation metrics being used, ideally with a description of why. --> |
|
|
|
[More Information Needed] |
|
|
|
### Results |
|
|
|
[More Information Needed] |
|
|
|
#### Summary |
|
|
|
|
|
|
|
## Model Examination |
|
|
|
You can visualize its attention heads using [BertViz](https://github.com/jessevig/bertviz) and attribution weights using [Captum](https://captum.ai/) - as [done in the base model](gbyuvd/chemselfies-base-bertmlm) in Interpretability section. |
|
|
|
|
|
### Model Architecture and Objective |
|
|
|
[More Information Needed] |
|
|
|
### Compute Infrastructure |
|
|
|
#### Hardware |
|
|
|
- Platform: Paperspace's Gradients |
|
- Compute: Free-P5000 (16 GB GPU, 30 GB RAM, 8 vCPU) |
|
|
|
#### Software |
|
|
|
- Python: 3.9.13 |
|
- Transformers: 4.42.4 |
|
- PyTorch: 2.3.1+cu121 |
|
- Accelerate: 0.32.0 |
|
- Datasets: 2.20.0 |
|
- Tokenizers: 0.19.1 |
|
- Ranger21: 0.0.1 |
|
- Selfies: 2.1.2 |
|
- RDKit: 2024.3.3 |
|
|
|
|
|
## Citation |
|
|
|
If you find this project useful in your research and wish to cite it, please use the following BibTex entry: |
|
|
|
```bibtex |
|
@software{chemfie_basebertmlm, |
|
author = {GP Bayu}, |
|
title = {{ChemFIE Base}: Pretraining A Lightweight BERT-like model on Molecular SELFIES}, |
|
url = {https://huggingface.co/gbyuvd/chemselfies-base-bertmlm}, |
|
version = {1.0}, |
|
year = {2024}, |
|
} |
|
``` |
|
|
|
## References |
|
[DeepSA](https://doi.org/10.1186/s13321-023-00771-3) |
|
|
|
```bibtex |
|
@article{Wang2023DeepSA, |
|
title={DeepSA: a deep-learning driven predictor of compound synthesis accessibility}, |
|
author={Wang, Shihang and Wang, Lin and Li, Fenglei and Bai, Fang}, |
|
journal={Journal of Cheminformatics}, |
|
volume={15}, |
|
pages={103}, |
|
year={2023}, |
|
month={Nov}, |
|
publisher={BioMed Central}, |
|
doi={10.1186/s13321-023-00771-3}, |
|
} |
|
|
|
``` |
|
|
|
## Contact & Support My Work |
|
|
|
G Bayu ([email protected]) |
|
|
|
This project has been quiet a journey for me, I’ve dedicated hours on this and I would like to improve myself, this model, and future projects. However, financial and computational constraints can be challenging. |
|
|
|
If you find my work valuable and would like to support my journey, please consider supporting me [here](https://ko-fi.com/gbyuvd). Your support will help me cover costs for computational resources, data acquisition, and further development of this project. Any amount, big or small, is greatly appreciated and will enable me to continue learning and explore more. |
|
|
|
Thank you for checking out this model, I am more than happy to receive any feedback, so that I can improve myself and the future model/projects I will be working on. |
