cmarkea
/

bloomz-3b-dpo-chat

@@ -7,10 +7,10 @@ language:
 - fr
 ---
-## bloomz-3b-dpo-chat Model Card
-### Model Overview
 The bloomz-3b-dpo-chat is a conversational model fine-tuned using Direct Preference Optimization (DPO) from the base bloomz-3b-sft-chat model. This model aims to
 provide high-quality conversational abilities in both English and French, leveraging the pre-trained strengths of its SFT (Supervised Fine-Tuning) predecessor.
@@ -19,24 +19,24 @@ provide high-quality conversational abilities in both English and French, levera
 ---
-### Model Description
 The bloomz-3b-dpo-chat model builds upon the solid foundation of the bloomz-3b-sft-chat, which is notable for its chatbot-specific pre-training and efficient
 tokenization strategy. The DPO fine-tuning process enhances the model's ability to generate more human-preferred responses in conversational contexts.
-### Multilingual Capabilities
 The model was initially trained on both French and English datasets, ensuring high efficiency and performance in these languages. Due to the DPO process and potential
 data type changes (from float16 to bfloat16), the model's multilingual capabilities might not be as robust as its SFT predecessor, but fine-tuning can help in restoring
 performance in other languages.
-### Model Applications
 This model is suitable for chatbot applications, customer service automation, and other conversational AI systems where bilingual (French and English) support is
 essential.
-### Dataset
 The bloomz-3b-dpo-chat model was trained using the [Anthropic/hh-rlhf](https://huggingface.co/datasets/Anthropic/hh-rlhf) dataset, which includes:
@@ -45,15 +45,7 @@ The bloomz-3b-dpo-chat model was trained using the [Anthropic/hh-rlhf](https://h
    - **Purpose:** To train preference models for Reinforcement Learning from Human Feedback (RLHF), not for supervised training of dialogue agents.
    - **Source:** Data from context-distilled language models, rejection sampling, and an iterated online process.
-2. **Red Teaming Data:**
-   - **Description:** Transcripts of conversations between human adversaries (red team members) and AI assistants, annotated for harmfulness.
-   - **Purpose:** To study and mitigate harmful behaviors in AI models, not for fine-tuning or preference modeling.
-   - **Content:** Transcripts, harmlessness scores, model parameters, success ratings, and red team attack descriptions.
-**Disclaimer:** The dataset contains sensitive and potentially upsetting content. It is intended for research to make AI models safer. Engage with caution.
-### Evaluation
 Evaluation of the model was conducted using the PoLL (Pool of LLM) technique, assessing performance on 100 French questions with scores aggregated from six evaluations
 (two per evaluator). The evaluators included GPT-4o, Gemini-1.5-pro, and Claude3.5-sonnet.
@@ -75,7 +67,7 @@ The bloomz-3b-dpo-chat model demonstrates improved performance over its SFT coun
 production environments.
-### Usage
 To utilize the bloomz-3b-dpo-chat model, format the prompt for chatbot interactions as follows:
 ```
@@ -100,7 +92,7 @@ result
 ```
-### Citation
 ```bibtex
 @online{DeBloomzChat,

 - fr
 ---
+# bloomz-3b-dpo-chat Model Card
+## Model Overview
 The bloomz-3b-dpo-chat is a conversational model fine-tuned using Direct Preference Optimization (DPO) from the base bloomz-3b-sft-chat model. This model aims to
 provide high-quality conversational abilities in both English and French, leveraging the pre-trained strengths of its SFT (Supervised Fine-Tuning) predecessor.
 ---
+## Model Description
 The bloomz-3b-dpo-chat model builds upon the solid foundation of the bloomz-3b-sft-chat, which is notable for its chatbot-specific pre-training and efficient
 tokenization strategy. The DPO fine-tuning process enhances the model's ability to generate more human-preferred responses in conversational contexts.
+## Multilingual Capabilities
 The model was initially trained on both French and English datasets, ensuring high efficiency and performance in these languages. Due to the DPO process and potential
 data type changes (from float16 to bfloat16), the model's multilingual capabilities might not be as robust as its SFT predecessor, but fine-tuning can help in restoring
 performance in other languages.
+## Model Applications
 This model is suitable for chatbot applications, customer service automation, and other conversational AI systems where bilingual (French and English) support is
 essential.
+## Dataset
 The bloomz-3b-dpo-chat model was trained using the [Anthropic/hh-rlhf](https://huggingface.co/datasets/Anthropic/hh-rlhf) dataset, which includes:
    - **Purpose:** To train preference models for Reinforcement Learning from Human Feedback (RLHF), not for supervised training of dialogue agents.
    - **Source:** Data from context-distilled language models, rejection sampling, and an iterated online process.
+## Evaluation
 Evaluation of the model was conducted using the PoLL (Pool of LLM) technique, assessing performance on 100 French questions with scores aggregated from six evaluations
 (two per evaluator). The evaluators included GPT-4o, Gemini-1.5-pro, and Claude3.5-sonnet.
 production environments.
+## Usage
 To utilize the bloomz-3b-dpo-chat model, format the prompt for chatbot interactions as follows:
 ```
 ```
+## Citation
 ```bibtex
 @online{DeBloomzChat,