instructinos for installing LLMs pretrained weights

README.md

@@ -48,11 +48,13 @@ cd /path/BioM3/weights
 ```
 
 Note: choose the desired BioM3 configuration/checkpoint, then install weights for each folder:
+- `/path/BioM3/weights/LLMs` # install ESM2 and PubMedBert pretrained wieghts for compiling PenCL
 - `/path/BioM3/weights/PenCL`
 - `/path/BioM3/weights/Facilitator` 
 - `/path/BioM3/weights/ProteoScribe`
 
 Each folder contains a `README.md` detailing the different model weight configurations. For benchmarking, the optimal configuration is:
+- `esm2_t33_650M_UR50D.pt`, `esm2_t33_650M_UR50D-contact-regression.pt`, and `BiomedNLP-BiomedBERT-base-uncased-abstract-fulltext`
 - `BioM3_PenCL_epoch20.bin`
 - `BioM3_Facilitator_epoch20.bin`
 - `BioM3_ProteoScribe_epoch20.bin`
@@ -68,10 +70,21 @@ This stage demonstrates how to perform inference using the **BioM3 PenCL model**
 
 Before running the model, ensure you have:
 - Configuration file: `stage1_config.json`
-- Pre-trained weights: `BioM3_PenCL_epoch20.bin`
+- Pre-trained weights: `BioM3_PenCL_epoch20.bin`, `esm2_t33_650M_UR50D.pt`, `esm2_t33_650M_UR50D-contact-regression.pt`, and `BiomedNLP-BiomedBERT-base-uncased-abstract-fulltext`. 
 
 ### Running the Model
 
+0. Change json configuration for Stage 1:
+
+```bash
+vim stage1_config.json
+
+# replace <working_directory> with your path
+"seq_model_path": "<working_directory>/BioM3/weights/LLMs/esm2_t33_650M_UR50D.pt"
+"text_model_path": "<working_directory>/weights/LLMs/BiomedNLP-BiomedBERT-base-uncased-abstract-fulltext",
+
+```
+
 1. Change directory to BioM3 repo:
 ```bash
 cd /path/BioM3 # /path/ where is the location to the cloned BioM3 repo

stage1_config.json

@@ -4,8 +4,8 @@
     "tb_logger_path": "None",
     "tb_logger_folder": "None",
     "version_name": "None",
-    "model_checkpoint_path": "/project/andrewferguson/niksapraljak/Project_ProtARDM/logs/Stage1_final_models/checkpoints/Pretraining_PENCiL_45M/epoch=19-step=116600.ckpt",
-    "output_dict_path": "/project/ranganathanr/niksapraljak/BioM3_PDZ/outputs/output_dict.pt",
+    "model_checkpoint_path": "None",
+    "output_dict_path": "None",
     "valid_size": 0.2,
     "epochs": 10,
     "acc_grad_batches": 1,
@@ -44,7 +44,5 @@
     "bLM_n_layers_to_finetune": 1,
     "proj_embedding_dim": 512,
     "dropout": 0.1,
-    "head_lr": 0.0005,
-    "inference_data_path": "/project/ranganathanr/niksapraljak/BioM3_PDZ/data/test_prompts_PDZ_swissprot_pfam_dataset.csv",
-    "inference_output_path": "/project/ranganathanr/niksapraljak/BioM3_PDZ/outputs/Stage1_test_prompts_PDZ.pt"
+    "head_lr": 0.0005
 }

weights/README.md

@@ -2,6 +2,7 @@
 
 This folder contains the pre-trained weights for the **BioM3** project models. The weights are stored as `.bin` files for different components of the BioM3 pipeline:
 
+0. **LLMs**: Pre-trained weights for compiling PenCL model. 
 1. **PenCL**: Pre-trained weights for the PenCL model (Stage 1).
 2. **Facilitator**: Pre-trained weights for the Facilitator model (Stage 2).
 3. **ProteoScribe**: Pre-trained weights for the ProteoScribe model (Stage 3).