Update README.md

TODO change value in table vidore when vidore/tatdqa_test ready

README.md

@@ -1,3 +1,69 @@
----
-license: apache-2.0
----
+---
+license: apache-2.0
+---
+# MonoQwen2-VL-2B-LoRA-Reranker
+
+## Model Overview
+The **MonoQwen2-VL-2B-LoRA-Reranker** is a fine-tuned version of the Qwen2-VL-2B model, optimized for reranking image-query relevance. It is built to process visual and text data and generate binary relevance scores. This model can be used in scenarios where reranking image relevance is crucial, such as document analysis and image-based search tasks.
+
+## How to Use the Model
+Below is a quick example to rerank a single image against a user query using this model:
+
+```python
+import torch
+from PIL import Image
+from transformers import AutoProcessor, Qwen2VLForConditionalGeneration
+
+# Load processor and model
+processor = AutoProcessor.from_pretrained("Qwen/Qwen2-VL-2B-Instruct")
+model = Qwen2VLForConditionalGeneration.from_pretrained("lightonai/MonoQwen2-VL-2B-LoRA-Reranker")
+
+# Define the query and the image
+query = "What is the value of the thing in the document"
+image = Image.open("path_to_image.jpg")
+
+# Prepare the inputs
+prompt = f"Assert the relevance of the previous image document to the following query, answer True or False. The query is: {query}"
+inputs = processor(text=prompt, images=image, return_tensors="pt")
+
+# Run the model and obtain results
+with torch.no_grad():
+    outputs = model(**inputs)
+    logits = outputs.logits
+    logits_for_last_token = logits[:, -1, :]
+    true_token_id = processor.tokenizer.convert_tokens_to_ids("True")
+    false_token_id = processor.tokenizer.convert_tokens_to_ids("False")
+    relevance_score = torch.softmax(logits_for_last_token[:, [true_token_id, false_token_id]], dim=-1)
+
+# Print the True/False probabilities
+true_prob = relevance_score[:, 0].item()
+false_prob = relevance_score[:, 1].item()
+
+print(f"True probability: {true_prob}, False probability: {false_prob}")
+```
+
+This example demonstrates how to use the model to assess the relevance of an image with respect to a query. It outputs the probability that the image is relevant ("True") or not relevant ("False").
+
+## Performance Metrics
+
+The model has been evaluated on [ViDoRe Benchmark](https://huggingface.co/spaces/vidore/vidore-leaderboard), by retrieving 10 elements with [MrLight_dse-qwen2-2b-mrl-v1](https://huggingface.co/MrLight/dse-qwen2-2b-mrl-v1) and reranking them. The table below summarizes its `ndcg@5` scores:
+
+| Dataset                                           | NDCG@5 Before Reranking  | NDCG@5 After Reranking |
+|---------------------------------------------------|--------------------------|------------------------|
+| **Mean**                                          | 87.6                     | **91.8**               |
+| vidore/arxivqa_test_subsampled                    | 85.6                     | 89.01                  |
+| vidore/docvqa_test_subsampled                     | 57.1                     | 59.71                  |
+| vidore/infovqa_test_subsampled                    | 88.1                     | 93.49                  |
+| vidore/tabfquad_test_subsampled                   | 93.1                     | 95.96                  |
+| vidore/shiftproject_test                          | 82.0                     | 92.98                  |
+| vidore/syntheticDocQA_artificial_intelligence_test| 97.5                     | 100.00                 |
+| vidore/syntheticDocQA_energy_test                 | 92.9                     | 97.65                  |
+| vidore/syntheticDocQA_government_reports_test     | 96.0                     | 98.04                  |
+| vidore/syntheticDocQA_healthcare_industry_test    | 96.4                     | 99.27                  |
+
+
+
+
+## License
+
+This LoRA model is licensed under the Apache 2.0 license.