README.md

metadata

title: medicalaiapp
app_file: app.py
sdk: gradio
sdk_version: 5.31.0

Medical VLM with SAM-2 and CheXagent

A comprehensive medical imaging analysis tool that combines:

• Qwen-VLM for medical visual question answering
• SAM-2 (Segment Anything Model 2) for automatic medical image segmentation
• CheXagent for structured chest X-ray report generation

Features

1. Medical Q&A: Ask questions about medical images using the Qwen-VLM model
2. Automatic Masking: Segment medical images automatically using SAM-2
3. Structured Report Generation: Generate detailed chest X-ray reports using CheXagent
4. Visual Grounding: Locate specific findings in medical images

Setup

1. Clone the repository:

git clone https://github.com/pascal-maker/medicalvlm.git
cd medicalvlm

2. Create and activate a virtual environment:

python -m venv chexagent_env
source chexagent_env/bin/activate  # On Windows: chexagent_env\Scripts\activate

3. Install dependencies:

pip install -r requirements.txt

4. Download required model checkpoints:

• SAM-2 checkpoint: Place in checkpoints/sam2.1_hiera_large.pt
• Other model weights will be downloaded automatically on first run

Usage

Run the Gradio interface:

python app.py

The web interface will be available at http://localhost:7860

Requirements

• Python 3.8+
• PyTorch
• CUDA-compatible GPU (recommended)
• See requirements.txt for full list of dependencies

License

[Your chosen license]

Acknowledgments

• Qwen-VLM
• SAM-2
• CheXagent