avemio
/

German-RAG-NEMO-12B-ORPO-HESSIAN-AI

@@ -3,6 +3,7 @@ license: mit
 datasets:
 - avemio/GRAG-CPT-HESSIAN-AI
 - avemio/GRAG-SFT-ShareGPT-HESSIAN-AI
 language:
 - en
 - de
@@ -28,7 +29,7 @@ tags:
 **GRAG** (**G**erman **R**etrieval **A**ugmented **G**eneration) models are designed for the German-speaking market, enabling innovation and AI solutions to drive German research collaboration in business-focused Generative AI by 2025
-Our GRAG-LLAMA-SFT model are trained on this **[GRAG-SFT](https://huggingface.co/datasets/avemio/GRAG-SFT-ShareGPT-HESSIAN-AI) dataset.**
 ## Model Details
@@ -133,37 +134,75 @@ Four evaluation metrics were employed across all subsets: language quality, over
 -   **Overall score:** This metric combined the results from the previous three metrics, offering a comprehensive evaluation of the model's capabilities across all subsets.
-| Metric                                    | [Vanila-Mistral-Nemo-Instruct-2407](https://huggingface.co/mistralai/Mistral-Nemo-Instruct-2407) | **[GRAG-NEMO-SFT](https://huggingface.co/avemio/GRAG-NEMO-12B-SFT-HESSIAN-AI)** | [GRAG-NEMO-ORPO](https://huggingface.co/avemio/GRAG-NEMO-12B-ORPO-HESSIAN-AI) | [GRAG-NEMO-MERGED]() | GPT-3.5-TURBO |
 |------------------------------------------|---------------------------------------------------------------------------------|--------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------|-----------------------------|----------------|
-| Average Language Quality             | 85.88                                                                          | **89.61**                                                                     | 89.1                                                                                          |                             |                |
 | **OVERALL SCORES (weighted):**       |                                                                            |                                                                            |                                                                                           |                             |               |
-| extraction_recall       | 35.2                                                                           | **52.3**                                                                           | 48.8                                                                                          |                             |                |
-| qa_multiple_references | 65.3                                                                           | **71.0**                                                                           | 74.0                                                                                          |                             |                |
-| qa_without_time_difference | 71.5                                                                           | **85.6**                                                                           | 85.6                                                                                          |                             |                |
-| qa_with_time_difference | 65.3                                                                           | **87.9**                                                                           | 85.4                                                                                          |                             |                |
-| reasoning              | 69.4                                                                           | **71.5**                                                                           | 73.4                                                                                          |                             |                |
-| relevant_context       | 71.3                                                                           | **69.1**                                                                           | 65.5                                                                                          |                             |                |
-| summarizations         | 73.8                                                                           | **81.6**                                                                           | 80.3                                                                                          |                             |                |
 ## Model Details
 ### Data
-For training data details, please see the [GRAG-SFT-Dataset](https://huggingface.co/datasets/avemio/GRAG-SFT-ShareGPT-HESSIAN-AI) documentation.
-#### Description
-The SFT tasks represent a focused approach to enhance model capabilities through specialized RAG examples. Most of these tasks were developed using synthetically enhanced data derived from the German Wikipedia, accessed through Cohere's prepared dataset on HuggingFace (licensed CC-BY-SA 4.0). This data was structured in a training knowledge graph where Question-Answer nodes were connected to both relevant and irrelevant Context nodes from the same Wikipedia page, creating a rich and challenging network of relationships for training. The only exceptions are the function calling dataset, which was derived and extended from Salesforce's XLAM Function calling dataset by including function call results and final answer generation, and the reasoning task which synthetic generation was inspired by the Paper from Tencent ([“Scaling Synthetic Data Creation with 1,000,000,000 Personas”](https://arxiv.org/abs/2406.20094)), to generate a diverse set of reasoning tasks across various domains.
-This comprehensive set of SFT tasks ensures the model develops robust capabilities across a wide range of practical applications while maintaining consistent output formats and clear communication patterns. Each task type has been carefully designed to address specific business needs while maintaining high standards of accuracy and reliability, making them valuable tools for organizations looking to enhance their information processing and knowledge management capabilities.
-#### Task Instruction Format
-The implementation of these SFT tasks follows a carefully structured format designed for consistency and clarity. Each task begins with comprehensive system instructions often wrapped in XML tags that meta-define expected inputs, outputs, constraints, and example interactions. This standardization enables clear communication between the model and users while ensuring reliable results.
-The context information utilized in these tasks is provided in a standardized JSON structure, including unique identifiers, source text, timestamps where relevant, and task-specific metadata. This format was specifically chosen to allow seamless integration with retrieved data from RAG systems, eliminating the need for additional formatting steps in production environments.
-Source references are handled through a consistent system of numerical indices for context references, JSON-formatted citation markers, and clear time-difference notifications when temporal aspects are relevant. This systematic approach to referencing ensures traceability and reliability in the model's responses.
-The implementation of these tasks within RAG systems can significantly improve organizational efficiency by reducing manual processing time, ensuring consistency in information handling, improving accuracy in data extraction and analysis, and enabling faster decision-making through better information access.
 ### Architecture
-| Parameter            | GRAG-NEMO-SFT                                                                                   |
 |-----------------------|-----------------------------------------------------------------------------------------------|
 | **d_model**          | 5120                                                                                         |
 | **num heads**        | 32                                                                                            |
@@ -181,7 +220,7 @@ The implementation of these tasks within RAG systems can significantly improve o
 ### Hyperparameters
-| Parameter                 | GRAG-NEMO-SFT       |
 |---------------------------|--------------------|
 | **warmup steps**          | 50                |
 | **peak LR**               | 5.0E-07           |
@@ -192,19 +231,19 @@ The implementation of these tasks within RAG systems can significantly improve o
 ## Environmental Impact
-GRAG-NEMO-SFT, running on NVIDIA A100 with 8 GPUs for 5 days, has an approximate power consumption as follows:
 It's important to note that the actual power consumption may vary depending on the specific workload and operational conditions. For accurate power consumption measurements, using dedicated power monitoring tools is recommended.
 | Model          | GPU Type            | Power Consumption From GPUs |
 |----------------|---------------------|-----------------------------|
-| GRAG-NEMO-SFT   | A100 ([Hessian AI supercomputer](https://hessian.ai/de/)) | 0.288 MWh                    |
 ## Bias, Risks, and Limitations
 Like any base language model or fine-tuned model without safety filtering, it is relatively easy for a user to prompt these models to generate harmful and generally sensitive content.
 Such content can also be produced unintentionally, especially in the case of bias, so we recommend users consider the risks of applications of this technology.
-Otherwise, many facts from GRAG-NEMO-SFT or any LLM will often not be true, so they should be checked.

 datasets:
 - avemio/GRAG-CPT-HESSIAN-AI
 - avemio/GRAG-SFT-ShareGPT-HESSIAN-AI
+- avemio/GRAG-ORPO-ShareGPT-HESSIAN-AI
 language:
 - en
 - de
 **GRAG** (**G**erman **R**etrieval **A**ugmented **G**eneration) models are designed for the German-speaking market, enabling innovation and AI solutions to drive German research collaboration in business-focused Generative AI by 2025
+Our GRAG-LLAMA-SFT model are trained on this **[GRAG-ORPO](https://huggingface.co/datasets/avemio/GRAG-ORPO-ShareGPT-HESSIAN-AI) dataset.**
 ## Model Details
 -   **Overall score:** This metric combined the results from the previous three metrics, offering a comprehensive evaluation of the model's capabilities across all subsets.
+| Metric                                    | [Vanila-Mistral-Nemo-Instruct-2407](https://huggingface.co/mistralai/Mistral-Nemo-Instruct-2407) | [GRAG-NEMO-SFT](https://huggingface.co/avemio/GRAG-NEMO-12B-SFT-HESSIAN-AI) | **[GRAG-NEMO-ORPO](https://huggingface.co/avemio/GRAG-NEMO-12B-ORPO-HESSIAN-AI)** | [GRAG-NEMO-MERGED]() | GPT-3.5-TURBO |
 |------------------------------------------|---------------------------------------------------------------------------------|--------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------|-----------------------------|----------------|
+| Average Language Quality             | 85.88                                                                          | 89.61                                                                     | **89.1**                                                                                          |                             |                |
 | **OVERALL SCORES (weighted):**       |                                                                            |                                                                            |                                                                                           |                             |               |
+| extraction_recall       | 35.2                                                                           | 52.3                                                                           | **48.8**                                                                                          |                             |                |
+| qa_multiple_references | 65.3                                                                           | 71.0                                                                           | **74.0**                                                                                          |                             |                |
+| qa_without_time_difference | 71.5                                                                           | 85.6                                                                           | **85.6**                                                                                          |                             |                |
+| qa_with_time_difference | 65.3                                                                           | 87.9                                                                           | **85.4**                                                                                          |                             |                |
+| reasoning              | 69.4                                                                           | 71.5                                                                           | **73.4**                                                                                          |                             |                |
+| relevant_context       | 71.3                                                                           | 69.1                                                                           | **65.5**                                                                                          |                             |                |
+| summarizations         | 73.8                                                                           | 81.6                                                                           | **80.3**                                                                                          |                             |                |
 ## Model Details
 ### Data
+For training data details, please see the [GRAG-ORPO-Dataset](https://huggingface.co/datasets/avemio/GRAG-ORPO-ShareGPT-HESSIAN-AI) documentation.
+The ORPO Tasks Dataset represents a specialized collection for fine-tuning language models with a focus on RAG-specific capabilities.
+The subsets can be for this training step are derived from 3 different sources:
+- **SauerkrautLM Preference Datasets**:
+  - [SauerkrautLM-Fermented-GER-DPO](https://huggingface.co/datasets/VAGOsolutions/SauerkrautLM-Fermented-GER-DPO):  is a specialized dataset designed for training language models in function calling irrelevance detection using Preference Optimization. The dataset consists of 2,000 carefully evaluated instruction-response pairs, specifically curated to help models recognize situations where function calls are unnecessary and direct responses are more appropriate.
+  - [SauerkrautLM-Fermented-Irrelevance-GER-DPO](https://huggingface.co/datasets/VAGOsolutions/SauerkrautLM-Fermented-Irrelevance-GER-DPO): is a high-quality German instruction-response dataset specifically designed for Preference Optimization training. The dataset consists of 3,305 instruction-response pairs. Rather than being merged from existing German datasets, it was carefully created through a sophisticated augmentation process, transforming curated English instructions and responses into culturally adapted German content. Each pair includes comprehensive quality metrics and rejected responses for Preference training.
+- **Hard Reasoning DE & EN**: Synthetic generation inspired by Tencent's ([“Scaling Synthetic Data Creation with 1,000,000,000 Personas”](https://arxiv.org/abs/2406.20094)).
+- **Multi-Turn-QA**: Developed by Avemio AG, this dataset builds upon and enhances the German Wikipedia dump provided by Cohere ([wikipedia-22-12-de-embeddings](https://huggingface.co/datasets/Cohere/wikipedia-22-12-de-embeddings)), expanding it with synthetic examples and structured tasks to create a robust training resource.
+### Data Subsets
+| Subset | Examples per Task |
+|-------|------------------|
+| SauerkrautLM-Fermented-GER-DPO | 3.31k |
+| SauerkrautLM-Fermented-Irrelevance-GER-DPO | 2k |
+| hard-reasoning-de | 3.19k |
+| hard-reasoning-en | 1.97k |
+| multi-turn-qa | 3.2k |
+### Source Data: SauerkrautLM
+[SauerkrautLM-Fermented-GER-DPO](https://huggingface.co/datasets/VAGOsolutions/SauerkrautLM-Fermented-GER-DPO)
+[SauerkrautLM-Fermented-Irrelevance-GER-DPO](https://huggingface.co/datasets/VAGOsolutions/SauerkrautLM-Fermented-Irrelevance-GER-DPO)
+### Source Data: Hard-Reasoning DE & EN
+- Base: ([proj-Persona/PersonaHub](https://huggingface.co/datasets/proj-persona/PersonaHub))
+- Enhancement: Synthetic data generation by Avemio AG
+- Quality: Automatic validation and curation of examples by Open Source LLM's
+### Methodology: Reasoning-DE & Reasoning-EN
+- Providing Persona Descriptions and rewriting in a similar style with a different focus area and name in german/english language
+- Generating Simple Logical Problems out of Persona-specific Views & Language.
+- Generating Approaches, Thinking-Steps & Solutions separately verified by Llama-3.1-405B-Instruct
+- Quality assurance and validation
+### Source Data: Multi-Turn-QA
+- Base: ([cohere/wikipedia-22-12-de-embeddings](https://huggingface.co/datasets/Cohere/wikipedia-22-12-de-embeddings))
+- Enhancement: Synthetic data generation by Avemio AG
+- Quality: Automatic validation and curation of examples by Open Source LLM's
+### Methodology: Multi-Turn-QA
+1. Extraction of base content from German Wikipedia
+2. Enhancement through synthetic example generation
+3. Structure addition for specific task types
+4. Quality assurance and validation
 ### Architecture
+| Parameter            | GRAG-NEMO-ORPO                                                                                   |
 |-----------------------|-----------------------------------------------------------------------------------------------|
 | **d_model**          | 5120                                                                                         |
 | **num heads**        | 32                                                                                            |
 ### Hyperparameters
+| Parameter                 | GRAG-NEMO-ORPO       |
 |---------------------------|--------------------|
 | **warmup steps**          | 50                |
 | **peak LR**               | 5.0E-07           |
 ## Environmental Impact
+GRAG-NEMO-ORPO, running on NVIDIA A100 with 8 GPUs for 5 days, has an approximate power consumption as follows:
 It's important to note that the actual power consumption may vary depending on the specific workload and operational conditions. For accurate power consumption measurements, using dedicated power monitoring tools is recommended.
 | Model          | GPU Type            | Power Consumption From GPUs |
 |----------------|---------------------|-----------------------------|
+| GRAG-NEMO-NEMO   | A100 ([Hessian AI supercomputer](https://hessian.ai/de/)) | 0.288 MWh                    |
 ## Bias, Risks, and Limitations
 Like any base language model or fine-tuned model without safety filtering, it is relatively easy for a user to prompt these models to generate harmful and generally sensitive content.
 Such content can also be produced unintentionally, especially in the case of bias, so we recommend users consider the risks of applications of this technology.
+Otherwise, many facts from GRAG-NEMO-ORPO or any LLM will often not be true, so they should be checked.