intfloat commited on
Commit
b620de4
·
verified ·
1 Parent(s): 519b6b5

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +7 -12
README.md CHANGED
@@ -11,11 +11,14 @@ language:
11
  - pl
12
  - tr
13
  - fr
 
14
  license: mit
 
15
  ---
 
16
  ## mmE5-mllama-11b-instruct
17
 
18
- [mmE5: Improving Multimodal Multilingual Embeddings via High-quality Synthetic Data](https://arxiv.org/abs/2502.08468.pdf). Haonan Chen, Liang Wang, Nan Yang, Yutao Zhu, Ziliang Zhao, Furu Wei, Zhicheng Dou, arXiv 2024
19
 
20
  This model is trained based on [Llama-3.2-11B-Vision](https://huggingface.co/meta-llama/Llama-3.2-11B-Vision).
21
 
@@ -41,7 +44,7 @@ pip install -r requirements.txt
41
 
42
  Then you can enter the directory to run the following command.
43
  ```python
44
- from transformers import MllamaForConditionalGeneration, AutoProcessor, AutoConfig
45
  import torch
46
  from PIL import Image
47
 
@@ -61,17 +64,9 @@ model_name = "intfloat/mmE5-mllama-11b-instruct"
61
 
62
  # Load Processor and Model
63
  processor = AutoProcessor.from_pretrained(model_name)
64
- processor.tokenizer.padding_side = "right"
65
-
66
- config = AutoConfig.from_pretrained(model_name)
67
- if hasattr(config, 'use_cache'):
68
- config.use_cache = False
69
- config.padding_side = "right"
70
  model = MllamaForConditionalGeneration.from_pretrained(
71
- model_name, config=config,
72
- torch_dtype=torch.bfloat16
73
  ).to("cuda")
74
- model.padding_side = "right"
75
  model.eval()
76
 
77
  # Image + Text -> Text
@@ -118,4 +113,4 @@ print(string, '=', compute_similarity(qry_output, tgt_output))
118
  journal={arXiv preprint arXiv:2502.08468},
119
  year={2025}
120
  }
121
- ```
 
11
  - pl
12
  - tr
13
  - fr
14
+ library_name: transformers
15
  license: mit
16
+ pipeline_tag: image-feature-extraction
17
  ---
18
+
19
  ## mmE5-mllama-11b-instruct
20
 
21
+ [mmE5: Improving Multimodal Multilingual Embeddings via High-quality Synthetic Data](https://arxiv.org/abs/2502.08468.pdf). Haonan Chen, Liang Wang, Nan Yang, Yutao Zhu, Ziliang Zhao, Furu Wei, Zhicheng Dou, arXiv 2025
22
 
23
  This model is trained based on [Llama-3.2-11B-Vision](https://huggingface.co/meta-llama/Llama-3.2-11B-Vision).
24
 
 
44
 
45
  Then you can enter the directory to run the following command.
46
  ```python
47
+ from transformers import MllamaForConditionalGeneration, AutoProcessor
48
  import torch
49
  from PIL import Image
50
 
 
64
 
65
  # Load Processor and Model
66
  processor = AutoProcessor.from_pretrained(model_name)
 
 
 
 
 
 
67
  model = MllamaForConditionalGeneration.from_pretrained(
68
+ model_name, torch_dtype=torch.bfloat16
 
69
  ).to("cuda")
 
70
  model.eval()
71
 
72
  # Image + Text -> Text
 
113
  journal={arXiv preprint arXiv:2502.08468},
114
  year={2025}
115
  }
116
+ ```