Upload README.md
Browse files
README.md
CHANGED
|
@@ -1,3 +1,30 @@
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
| 3 |
+
datasets:
|
| 4 |
+
- ai4colonoscopy/ColonINST-v1
|
| 5 |
+
language:
|
| 6 |
+
- en
|
| 7 |
+
metrics:
|
| 8 |
+
- accuracy
|
| 9 |
+
base_model:
|
| 10 |
+
- microsoft/phi-1_5
|
| 11 |
+
library_name: adapter-transformers
|
| 12 |
+
pipeline_tag: image-text-to-text
|
| 13 |
+
tags:
|
| 14 |
+
- medical
|
| 15 |
+
- colonoscopy
|
| 16 |
+
- polyp
|
| 17 |
---
|
| 18 |
+
|
| 19 |
+
# ColonGPT (A colonoscopy-specific multimodal Language Model)
|
| 20 |
+
<p align="center">
|
| 21 |
+
<img src="./assert/web_ui_stg1.gif" width="666px"/> <br />
|
| 22 |
+
<em>The Gradio Web UI allows you to use our examples or upload your images for inference.</em>
|
| 23 |
+
</p>
|
| 24 |
+
|
| 25 |
+
📖 [Paper](https://arxiv.org/abs/2410.17241) | 🏠 [Home](https://github.com/ai4colonoscopy/IntelliScope)
|
| 26 |
+
|
| 27 |
+
> This is the weight of the pre-alignment stage of ColonGPT-v1.
|
| 28 |
+
|
| 29 |
+
|
| 30 |
+
Our ColonGPT is a standard multimodal language model, which contains four basic components: a language tokenizer, an visual encoder (🤗 [SigLIP-SO](https://huggingface.co/google/siglip-so400m-patch14-384)), a multimodal connector, and a language model (🤗 [Phi1.5](https://huggingface.co/microsoft/phi-1_5)). In this huggingface page, we provide a quick start for convenient of new users. For further details about ColonGPT, we highly recommend visiting our [homepage](https://github.com/ai4colonoscopy/IntelliScope). There, you'll find comprehensive usage instructions for our model and the latest advancements in intelligent colonoscopy technology.
|