Adrian's picture
4 11

Adrian

adriansanz

AI & ML interests

None yet

Recent Activity

reacted to singhsidhukuldeep's post with šŸ”„ 9 days ago
O1 Embedder: Transforming Retrieval Models with Reasoning Capabilities Researchers from University of Science and Technology of China and Beijing Academy of Artificial Intelligence have developed a novel retrieval model that mimics the slow-thinking capabilities of reasoning-focused LLMs like OpenAI's O1 and DeepSeek's R1. Unlike traditional embedding models that directly match queries with documents, O1 Embedder first generates thoughtful reflections about the query before performing retrieval. This two-step process significantly improves performance on complex retrieval tasks, especially those requiring intensive reasoning or zero-shot generalization to new domains. The technical implementation is fascinating: - The model integrates two essential functions: Thinking and Embedding - It uses an "Exploration-Refinement" data synthesis workflow where initial thoughts are generated by an LLM and refined by a retrieval committee - A multi-task training method fine-tunes a pre-trained LLM to generate retrieval thoughts via behavior cloning while simultaneously learning embedding capabilities through contrastive learning - Memory-efficient joint training enables both tasks to share encoding results, dramatically increasing batch size The results are impressive - O1 Embedder outperforms existing methods across 12 datasets in both in-domain and out-of-domain scenarios. For example, it achieves a 3.9% improvement on Natural Questions and a 3.0% boost on HotPotQA compared to models without thinking capabilities. This approach represents a significant paradigm shift in retrieval technology, bridging the gap between traditional dense retrieval and the reasoning capabilities of large language models. What do you think about this approach? Could "thinking before retrieval" transform how we build search systems?
updated a Space 13 days ago
adriansanz/test
published a Space 13 days ago
adriansanz/test
View all activity

Organizations

Hugging Face Discord Community's profile picture

adriansanz's activity

reacted to singhsidhukuldeep's post with šŸ”„ 9 days ago
view post
Post
3452
O1 Embedder: Transforming Retrieval Models with Reasoning Capabilities

Researchers from University of Science and Technology of China and Beijing Academy of Artificial Intelligence have developed a novel retrieval model that mimics the slow-thinking capabilities of reasoning-focused LLMs like OpenAI's O1 and DeepSeek's R1.

Unlike traditional embedding models that directly match queries with documents, O1 Embedder first generates thoughtful reflections about the query before performing retrieval. This two-step process significantly improves performance on complex retrieval tasks, especially those requiring intensive reasoning or zero-shot generalization to new domains.

The technical implementation is fascinating:

- The model integrates two essential functions: Thinking and Embedding
- It uses an "Exploration-Refinement" data synthesis workflow where initial thoughts are generated by an LLM and refined by a retrieval committee
- A multi-task training method fine-tunes a pre-trained LLM to generate retrieval thoughts via behavior cloning while simultaneously learning embedding capabilities through contrastive learning
- Memory-efficient joint training enables both tasks to share encoding results, dramatically increasing batch size

The results are impressive - O1 Embedder outperforms existing methods across 12 datasets in both in-domain and out-of-domain scenarios. For example, it achieves a 3.9% improvement on Natural Questions and a 3.0% boost on HotPotQA compared to models without thinking capabilities.

This approach represents a significant paradigm shift in retrieval technology, bridging the gap between traditional dense retrieval and the reasoning capabilities of large language models.

What do you think about this approach? Could "thinking before retrieval" transform how we build search systems?
reacted to dylanebert's post with šŸ”„ about 1 month ago
reacted to cutechicken's post with ā¤ļø 3 months ago
view post
Post
2944
šŸš€ RAGOndevice: High-Performance Local AI Document Analysis Assistant
šŸ’« Core Value
RAGOndevice is a high-performance AI system running locally without cloud dependency. Using CohereForAI's optimized 7B model, it enables professional-grade document analysis on standard PCs. āœØ
šŸŒŸ Ondevice AI Advantages
1. šŸ”‹ Efficient Resource Utilization

šŸŽÆ Optimized 7B Model: Runs on standard PCs
āš” Local Processing: Instant response without cloud
šŸ’» Low-Spec Compatible: Performs well on regular GPUs
šŸ”„ Optimized Memory: Ensures stable operation

2. šŸ›”ļø Data Security & Cost Efficiency

šŸ”’ Complete Privacy: No external data transmission
šŸŒ Offline Operation: No internet required
šŸ’° No Subscription: One-time installation
āš™ļø Resource Optimization: Uses existing hardware

šŸŽ® Key Features
1. šŸ“Š Powerful Document Analysis

šŸ“ Multi-Format Support: TXT, CSV, PDF, Parquet
šŸ§  Intelligent Analysis: Automatic structure recognition
šŸ‘ļø OCR Support: Advanced PDF text extraction
šŸ’¬ Real-time Chat: Natural language interaction

2. šŸ” Local RAG System

šŸŽÆ Efficient Search: TF-IDF based local search
šŸ§© Context Understanding: Accurate information retrieval
šŸ“š Wikipedia Integration: Rich background knowledge

šŸŽÆ Use Cases

šŸ¢ Enterprise: Secure confidential document processing
šŸ”¬ Personal Research: Private data analysis
šŸ“š Education: Personal learning material analysis
šŸ’» Development: Local codebase analysis

ā­ Differentiators

šŸƒā€ā™‚ļø Independent Operation: Zero cloud dependency
āš” Instant Response: No network latency
šŸ” Complete Security: Full data control
šŸ’Ž Cost Efficiency: No ongoing costs

šŸ”® Future Plans

šŸš€ Enhanced model optimization
šŸ“š Local knowledge base expansion
āš” Hardware optimization
šŸ“ Extended file support


šŸŒŸ RAGOndevice democratizes high-performance AI, providing the optimal local AI solution for security-sensitive environments. šŸš€

šŸ”„ Power of Local AI: Experience enterprise-grade AI capabilities right on your device!

VIDraft/RAGOndevice