Update README.md

README.md

@@ -2,7 +2,6 @@
 model-index:
 - name: notus-7b-dpo-lora
   results: []
-license: mit
 datasets:
 - argilla/ultrafeedback-binarized-avg-rating-for-dpo
 language:
@@ -10,36 +9,37 @@ language:
 base_model: alignment-handbook/zephyr-7b-sft-full
 library_name: transformers
 pipeline_tag: text-generation
+tags:
+- dpo
+- preference
+- ultrafeedback
+license: apache-2.0
 ---
 
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
+# Model Card for Notus 7B
 
+Notus is going to be a collection of fine-tuned models using DPO, similarly to Zephyr, but mainly focused
+on the Direct Preference Optimization (DPO) step, aiming to incorporate preference feedback into the LLMs
+when fine-tuning those. Notus models are intended to be used as assistants via chat-like applications, and 
+are evaluated with the MT-Bench and AlpacaEval benchmarks, to be directly compared with Zephyr fine-tuned models
+also using DPO.
 
 ## Model Details
 
 ### Model Description
 
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
+- **Developed by:** Argilla, Inc. (based on HuggingFace H4 and MistralAI previous efforts and amazing work)
+- **Shared by:** Argilla, Inc.
+- **Model type:** GPT-like 7B model DPO fine-tuned using LoRA
+- **Language(s) (NLP):** Mainly English
+- **License:** Apache 2.0 (same as Zephyr 7B SFT and Mistral 7B v0.1)
+- **Finetuned from model:** [`alignment-handbook/zephyr-7b-sft-full`](https://huggingface.co/alignment-handbook/zephyr-7b-sft-full)
 
 ### Model Sources [optional]
 
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
+- **Repository:** https://github.com/argilla-io/notus-7b-dpo
+- **Paper:** N/A
+- **Demo:** https://argilla-notus-chat-ui.hf.space/
 
 ## Uses
 
@@ -139,26 +139,7 @@ Use the code below to get started with the model.
 #### Summary
 
 
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
+## Technical Specifications
 
 ### Model Architecture and Objective
 
@@ -170,7 +151,7 @@ Carbon emissions can be estimated using the [Machine Learning Impact calculator]
 
 #### Hardware
 
-[More Information Needed]
+8 x A100 40GB
 
 #### Software
 
@@ -206,11 +187,4 @@ Carbon emissions can be estimated using the [Machine Learning Impact calculator]
 
 [More Information Needed]
 
-
 ## Training procedure
-
-
-### Framework versions
-
-
-- PEFT 0.6.1