Create README.md
Browse files
README.md
ADDED
@@ -0,0 +1,51 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
---
|
2 |
+
datasets:
|
3 |
+
- chart-misinformation-detection/MISCHA-QA-v1
|
4 |
+
language:
|
5 |
+
- en
|
6 |
+
tags:
|
7 |
+
- LLaVA
|
8 |
+
- misinformation
|
9 |
+
---
|
10 |
+
# Model Card for Snoopy 1.0
|
11 |
+
|
12 |
+
This model aims to detect visual manipulation in bar charts.
|
13 |
+
|
14 |
+
|
15 |
+
## Model Details
|
16 |
+
|
17 |
+
### Model Description
|
18 |
+
|
19 |
+
<!-- Provide a longer summary of what this model is. -->
|
20 |
+
|
21 |
+
- **Developed by:** Arif Syraj
|
22 |
+
- **Model type:** Multi-Modal LLM
|
23 |
+
- **Finetuned from model:** llava-1.6-mistral-7b
|
24 |
+
|
25 |
+
## Training Details
|
26 |
+
|
27 |
+
Finetuned with LoRA for 1 epoch on ~2700 images of misleading and non misleading bar charts
|
28 |
+
|
29 |
+
### Training Procedure
|
30 |
+
|
31 |
+
<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
|
32 |
+
learning_rate = 1e-5
|
33 |
+
bf16 = True
|
34 |
+
num_train_epochs = 1
|
35 |
+
optim = "adamw_torch"
|
36 |
+
per_device_train_batch_size = 3
|
37 |
+
gradient_accumulation_steps = 16
|
38 |
+
gradient_checkpointing = True
|
39 |
+
|
40 |
+
LoRA config:
|
41 |
+
rank = 32,
|
42 |
+
lora_alpha = 32,
|
43 |
+
Using rank stabilized lora
|
44 |
+
target_modules=[q_proj, out_proj, v_proj, k_proj, down_proj, up_proj, o_proj, gate_proj]
|
45 |
+
lora_dropout=0.05,
|
46 |
+
bias="none"
|
47 |
+
|
48 |
+
|
49 |
+
#### Training Hyperparameters
|
50 |
+
|
51 |
+
- **Training regime:** bf16 non-mixed precision
|