ColPali
Safetensors
English
vidore
manu commited on
Commit
7c446a5
·
verified ·
1 Parent(s): 730dded

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +5 -2
README.md CHANGED
@@ -1,6 +1,7 @@
1
  ---
2
  license: mit
3
  library_name: colpali
 
4
  language:
5
  - en
6
  tags:
@@ -13,6 +14,8 @@ It is a [PaliGemma-3B](https://huggingface.co/google/paligemma-3b-mix-448) exten
13
  It was introduced in the paper [ColPali: Efficient Document Retrieval with Vision Language Models](https://arxiv.org/abs/2407.01449) and first released in [this repository](https://github.com/ManuelFay/colpali)
14
 
15
  This version has right padding to fix unwanted tokens in the query encoding.
 
 
16
 
17
  ## Model Description
18
 
@@ -58,8 +61,8 @@ def main() -> None:
58
  """Example script to run inference with ColPali"""
59
 
60
  # Load model
61
- model_name = "vidore/colpali"
62
- model = ColPali.from_pretrained("google/paligemma-3b-mix-448", torch_dtype=torch.bfloat16, device_map="cuda").eval()
63
  model.load_adapter(model_name)
64
  processor = AutoProcessor.from_pretrained(model_name)
65
 
 
1
  ---
2
  license: mit
3
  library_name: colpali
4
+ base_model: vidore/colpaligemma-3b-mix-448-base
5
  language:
6
  - en
7
  tags:
 
14
  It was introduced in the paper [ColPali: Efficient Document Retrieval with Vision Language Models](https://arxiv.org/abs/2407.01449) and first released in [this repository](https://github.com/ManuelFay/colpali)
15
 
16
  This version has right padding to fix unwanted tokens in the query encoding.
17
+ It also stems from the fixed `vidore/colpaligemma-3b-mix-448-base` to guarantee deterministic projection layer initialization.
18
+
19
 
20
  ## Model Description
21
 
 
61
  """Example script to run inference with ColPali"""
62
 
63
  # Load model
64
+ model_name = "manu/colpali-v1.1"
65
+ model = ColPali.from_pretrained("vidore/colpaligemma-3b-mix-448-base", torch_dtype=torch.bfloat16, device_map="cuda").eval()
66
  model.load_adapter(model_name)
67
  processor = AutoProcessor.from_pretrained(model_name)
68