ChemGPT 19M

ChemGPT is based on the GPT-Neo model and was introduced in the paper Neural Scaling of Deep Chemical Models.

Model description

ChemGPT is a transformers model for generative molecular modeling, which was pretrained on the PubChem10M dataset.

Intended uses & limitations

How to use

You can use this model directly from the 🤗/transformers library.

Limitations and bias

This model was trained on a subset of molecules from PubChem. You can use this model to generate molecules, but it is mostly intended to be used for investigations of the effects of pre-training and fine-tuning on downstream datasets.

Training data

PubChem10M, a dataset of SMILES strings from PubChem, available via DeepChem.

Training procedure

Preprocessing

SMILES strings were converted to SELFIES using version 1.0.4 of the SELFIES library.

Pretraining

See code in the LitMatter repository.

BibTeX entry and citation info

@article{frey_soklaski_axelrod_samsi_gomez-bombarelli_coley_gadepally_2022, 
place={Cambridge}, title={Neural Scaling of Deep Chemical Models}, 
DOI={10.26434/chemrxiv-2022-3s512}, journal={ChemRxiv}, publisher={Cambridge Open Engage}, 
author={Frey, Nathan and Soklaski, Ryan and Axelrod, Simon and Samsi, Siddharth and Gomez-Bombarelli, Rafael and Coley, Connor and Gadepally, Vijay}, 
year={2022}} This content is a preprint and has not been peer-reviewed.

Frey, Nathan, Ryan Soklaski, Simon Axelrod, Siddharth Samsi, Rafael Gomez-Bombarelli, Connor Coley, and Vijay Gadepally. 
"Neural Scaling of Deep Chemical Models." ChemRxiv (2022). Print. This content is a preprint and has not been peer-reviewed.