ICMA version of galactica-125M for molecule captioning task (Mol2Cap) for paper "Large Language Models are In-Context Molecule Learners"
Notice: The input should contain 2 context examples and the cutoff length should be set to 2048 to ensure best performance.
A simple inference example:
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("phenixace/ICMA-Galactica-125M-M2C")
from transformers import AutoTokenizer
tk = AutoTokenizer.from_pretrained("phenixace/ICMA-Galactica-125M-M2C")
from transformers import GenerationConfig
text = """Generate a caption for the molecule: C[C@]12CCC(=O)C=C1CC[C@@H]3[C@@H]2C(=O)C[C@]4([C@H]3CCC4=O)C
Caption: The molecule is a 3-oxo Delta(4)-steroid that is androst-4-ene carrying three oxo-substituents at positions 3, 11 and 17. It has a role as an androgen, a human urinary metabolite, a marine metabolite and an EC 1.1.1.146 (11beta-hydroxysteroid dehydrogenase) inhibitor. It is a 3-oxo-Delta(4) steroid, a 17-oxo steroid, an androstanoid and an 11-oxo steroid. It derives from a hydride of an androstane.
Generate a caption for the molecule: C[C@]12CCC(=O)C=C1CC[C@@H]3[C@@H]2C(=O)C[C@]4([C@H]3CC[C@@H]4C(=O)CO)C
Caption: The molecule is an 11-oxo steroid that is corticosterone in which the hydroxy substituent at the 11beta position has been oxidised to give the corresponding ketone. It has a role as a human metabolite and a mouse metabolite. It is a 21-hydroxy steroid, a 3-oxo-Delta(4) steroid, a 20-oxo steroid, an 11-oxo steroid, a corticosteroid and a primary alpha-hydroxy ketone. It derives from a corticosterone.
Based on the above examples, analyse the similarities and differences between the examples and finally generate a caption for the molecule: C[C@]12CCC(=O)C=C1CC[C@@H]3[C@@H]2C(=O)C[C@]\\4([C@H]3CC/C4=C/C(=O)OC)C."""
generation_config = GenerationConfig(
do_sample=True,
temperature=0.7,
top_p=0.85,
top_k=40,
num_beams=1,
repetition_penalty=1.0,
pad_token_id=0,
)
inputs = tk(text, return_tensors="pt", return_token_type_ids=False)
outputs = model.generate(**inputs, return_dict_in_generate=True, output_scores=True, num_return_sequences=1, max_new_tokens=256, generation_config=generation_config)
# decode
decoded = tk.decode(outputs.sequences[0], skip_special_tokens=True)
print(decoded)
Paper Link: https://arxiv.org/abs/2403.04197
- Downloads last month
- 126
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.