Update README.md
Browse files
README.md
CHANGED
@@ -9,6 +9,31 @@ tags:
|
|
9 |
- llava
|
10 |
- lora
|
11 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
12 |
|
13 |
### Training hyperparameters
|
14 |
The following hyperparameters were used during training:
|
|
|
9 |
- llava
|
10 |
- lora
|
11 |
---
|
12 |
+
<style>
|
13 |
+
.img-responsive {
|
14 |
+
width: 50%;
|
15 |
+
height: auto;
|
16 |
+
}
|
17 |
+
</style>
|
18 |
+
|
19 |
+
### Conclusion
|
20 |
+
While significantly better at understanding and describing emotions and details in images compared to LLaVA-1.5-7b-hf, the fine-tuned model struggles with recognizing text.
|
21 |
+
|
22 |
+
### Train Loss
|
23 |
+

|
24 |
+
|
25 |
+
### Test
|
26 |
+
A comparative analysis of emoji in prompts, differents between the original model and its fine-tuned counterpart. </br>
|
27 |
+
Original Model:https://huggingface.co/llava-hf/llava-1.5-7b-hf/</br>
|
28 |
+
<img src="./images/original-01.JPG" alt="meme01" class="img-responsive">
|
29 |
+
<img src="./images/original-02.JPG" alt="meme02" class="img-responsive">
|
30 |
+
<img src="./images/original-03.JPG" alt="meme03" class="img-responsive">
|
31 |
+
|
32 |
+
|
33 |
+
Fine-tuned Lora Model:https://huggingface.co/REILX/llava-1.5-7b-hf-meme-lora</br>
|
34 |
+
<img src="./images/lora-01.JPG" alt="meme01" class="img-responsive">
|
35 |
+
<img src="./images/lora-02.JPG" alt="meme02" class="img-responsive">
|
36 |
+
<img src="./images/lora-03.JPG" alt="meme03" class="img-responsive">
|
37 |
|
38 |
### Training hyperparameters
|
39 |
The following hyperparameters were used during training:
|