ByteDance/Sa2VA-4B
Image-Text-to-Text
ā¢
Updated
ā¢
1.36k
ā¢
35
Umitcan Sahin PRO
ucsahin
AI & ML interests
Visual Language Models, Large Language Models, Vision Transformers
Recent Activity
liked
a model
4 days ago
ByteDance/Sa2VA-4B
liked
a model
4 days ago
microsoft/phi-4
liked
a model
4 days ago
ByteDance/Sa2VA-8B
Organizations
None yet
ucsahin's activity
liked
4
models
4 days ago
reacted to
m-ric's
post with š¤šš„
6 days ago
Post
4848
Since I published it on GitHub a few days ago,
Hugging Face's new agentic library ššŗš¼š¹š®š“š²š»šš has gathered nearly 4k stars š¤Æ
ā”ļø But we are just getting started on agents: so we are hiring an ML Engineer to join me and double down on this effort!
The plan is to build GUI agents: agents that can act on your computer with mouse & keyboard, like Claude Computer Use.
We will make it work better, and fully open. āØ
Sounds like something you'd like to do? Apply here š https://apply.workable.com/huggingface/j/AF1D4E3FEB/
Hugging Face's new agentic library ššŗš¼š¹š®š“š²š»šš has gathered nearly 4k stars š¤Æ
ā”ļø But we are just getting started on agents: so we are hiring an ML Engineer to join me and double down on this effort!
The plan is to build GUI agents: agents that can act on your computer with mouse & keyboard, like Claude Computer Use.
We will make it work better, and fully open. āØ
Sounds like something you'd like to do? Apply here š https://apply.workable.com/huggingface/j/AF1D4E3FEB/
- 3 replies
liked
2
datasets
7 days ago
liked
a
model
7 days ago
upvoted
2
collections
15 days ago
reacted to
singhsidhukuldeep's
post with š„
21 days ago
Post
2183
Exciting News in AI: JinaAI Releases JINA-CLIP-v2!
The team at Jina AI has just released a groundbreaking multilingual multimodal embedding model that's pushing the boundaries of text-image understanding. Here's why this is a big deal:
š Technical Highlights:
- Dual encoder architecture combining a 561M parameter Jina XLM-RoBERTa text encoder and a 304M parameter EVA02-L14 vision encoder
- Supports 89 languages with 8,192 token context length
- Processes images up to 512Ć512 pixels with 14Ć14 patch size
- Implements FlashAttention2 for text and xFormers for vision processing
- Uses Matryoshka Representation Learning for efficient vector storage
ā”ļø Under The Hood:
- Multi-stage training process with progressive resolution scaling (224ā384ā512)
- Contrastive learning using InfoNCE loss in both directions
- Trained on massive multilingual dataset including 400M English and 400M multilingual image-caption pairs
- Incorporates specialized datasets for document understanding, scientific graphs, and infographics
- Uses hard negative mining with 7 negatives per positive sample
š Performance:
- Outperforms previous models on visual document retrieval (52.65% nDCG@5)
- Achieves 89.73% image-to-text and 79.09% text-to-image retrieval on CLIP benchmark
- Strong multilingual performance across 30 languages
- Maintains performance even with 75% dimension reduction (256D vs 1024D)
šÆ Key Innovation:
The model solves the long-standing challenge of unifying text-only and multi-modal retrieval systems while adding robust multilingual support. Perfect for building cross-lingual visual search systems!
Kudos to the research team at Jina AI for this impressive advancement in multimodal AI!
The team at Jina AI has just released a groundbreaking multilingual multimodal embedding model that's pushing the boundaries of text-image understanding. Here's why this is a big deal:
š Technical Highlights:
- Dual encoder architecture combining a 561M parameter Jina XLM-RoBERTa text encoder and a 304M parameter EVA02-L14 vision encoder
- Supports 89 languages with 8,192 token context length
- Processes images up to 512Ć512 pixels with 14Ć14 patch size
- Implements FlashAttention2 for text and xFormers for vision processing
- Uses Matryoshka Representation Learning for efficient vector storage
ā”ļø Under The Hood:
- Multi-stage training process with progressive resolution scaling (224ā384ā512)
- Contrastive learning using InfoNCE loss in both directions
- Trained on massive multilingual dataset including 400M English and 400M multilingual image-caption pairs
- Incorporates specialized datasets for document understanding, scientific graphs, and infographics
- Uses hard negative mining with 7 negatives per positive sample
š Performance:
- Outperforms previous models on visual document retrieval (52.65% nDCG@5)
- Achieves 89.73% image-to-text and 79.09% text-to-image retrieval on CLIP benchmark
- Strong multilingual performance across 30 languages
- Maintains performance even with 75% dimension reduction (256D vs 1024D)
šÆ Key Innovation:
The model solves the long-standing challenge of unifying text-only and multi-modal retrieval systems while adding robust multilingual support. Perfect for building cross-lingual visual search systems!
Kudos to the research team at Jina AI for this impressive advancement in multimodal AI!
upvoted
a
collection
about 1 month ago
[bot] Conversion to Parquet
#1 opened 4 months ago
by
parquet-converter
upvoted
a
collection
about 1 month ago
reacted to
merve's
post with š„šš
about 2 months ago
Post
2191
The authors of ColPali trained a retrieval model based on SmolVLM š¤
vidore/colsmolvlm-alpha
TLDR;
- ColSmolVLM performs better than ColPali and DSE-Qwen2 on all English tasks
- ColSmolVLM is more memory efficient than ColQwen2 š
TLDR;
- ColSmolVLM performs better than ColPali and DSE-Qwen2 on all English tasks
- ColSmolVLM is more memory efficient than ColQwen2 š
liked
a
model
about 2 months ago
