Hugging Face's logo

shiym2000
/
MedM-VL-2D-3B-en

Safetensors
English
Model card Files Community
MedM-VL-2D-3B-en
File size: 2,632 Bytes 
17c2e81
 
 
 
 
 
411eb3d
17c2e81
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
 
---
license: apache-2.0
language:
- en
---

# MedM-VL-2D-3B-en

## Introduction

A medical LVLM, trained on **English** data, accepts text and **a single 2D medical image** as input, and text-based results as output. enabling tasks such as **report generation**, **medical VQA**, **referring expression comprehension**, **referring expression generation** and **image classification**.

Here are the evaluation results on **Uni-Med**:

<table>
  <tr>
    <td align="center"> Method </td>
    <td align="center"> medmnist_derma </td>
    <td align="center"> medmnist_organs </td>
    <td align="center"> medpix </td>
    <td align="center"> mimic </td>
    <td align="center"> pathvqa </td>
    <td align="center"> samed_identify </td>
    <td align="center"> samed_refer </td>
    <td align="center"> slake_identify </td>
    <td align="center"> slake_refer </td>
    <td align="center"> slakevqa </td>
  </tr>
  <tr>
    <td> Med-Flamingo </td>
    <td align="center"> 1.15 </td>
    <td align="center"> 8.90 </td>
    <td align="center"> 8.14 </td>
    <td align="center"> 23.25 </td>
    <td align="center"> 33.38 </td>
    <td align="center"> - </td>
    <td align="center"> - </td>
    <td align="center"> - </td>
    <td align="center"> - </td>
    <td align="center"> 21.51 </td>
  </tr>
  <tr>
    <td> RadFM </td>
    <td align="center"> 5.14 </td>
    <td align="center"> 18.90 </td>
    <td align="center"> - </td>
    <td align="center"> 6.81 </td>
    <td align="center"> 24.83 </td>
    <td align="center"> - </td>
    <td align="center"> - </td>
    <td align="center"> - </td>
    <td align="center"> - </td>
    <td align="center"> 81.66 </td>
  </tr>
  <tr>
    <td> LLaVA-Med </td>
    <td align="center"> 25.84 </td>
    <td align="center"> 66.80 </td>
    <td align="center"> <b>15.11</b> </td>
    <td align="center"> 20.43 </td>
    <td align="center"> 37.79 </td>
    <td align="center"> 45.83 </td>
    <td align="center"> 8.64 </td>
    <td align="center"> 27.21 </td>
    <td align="center"> 4.07 </td>
    <td align="center"> 33.69 </td>
  </tr>
  <tr>
    <td> MedM-VL-2D-3B-en </td>
    <td align="center"> <b>81.05</b> </td>
    <td align="center"> <b>72.14</b> </td>
    <td align="center"> 13.16 </td>
    <td align="center"> <b>22.63</b> </td>
    <td align="center"> <b>62.86</b> </td>
    <td align="center"> <b>70.97</b> </td>
    <td align="center"> <b>20.46</b> </td>
    <td align="center"> <b>68.94</b> </td>
    <td align="center"> <b>31.92</b> </td>
    <td align="center"> <b>84.45</b> </td>
  </tr>
</table>


## Quickstart

Please refer to [MedM-VL](https://github.com/MSIIP/MedM-VL).