Can CONCH v15 be used to encode text?

by Hailin123 - opened May 4

May 4

I used CONCH v15 to extract patch features. Can I use CONCH v15 to extract text features? Is there a related demo? Thanks

AI for Pathology Image Analysis Lab @ HMS / BWH org May 4

TITAN comes with text encoder that can be used for this purpose (https://huggingface.co/MahmoodLab/TITAN/blob/main/modeling_titan.py#L35).
You can also check zero-shot notebook example (https://github.com/mahmoodlab/TITAN/blob/main/notebooks/zeroshot_demo.ipynb).
Note that this text encoder was initialized with CONCHv1.5 text encoder, but was finetuned further as part of the vision-language pretraining.

Hope this helps,
Andrew

andrewsong90 changed discussion status to closed May 4

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment